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(57) Abstract 

A force feedback interface device (14X coupled to a 
host computer (18). inclades a manipulandimi (12). such as 
a mouse, that is moveable in a local workspace (24). A cursor 
position (222) tn the display frame (28) is reported to the 
host computer (18) derived from a reference position (204) 
of tlie mouse (12) hi the local frame (30), and die host (18) 
displays die cursor (180); for example, the cursor position 
(222) may be scaled by a ballistics algorithm (206) based on 
mouse velocity to allow fine positioning or coarse motion of 
the cursor (180). A force (216) is ou^ut on tbc mouse (12X 
die force (216) bemg determined based on mouse reference 
data or cuisor ballistic data, depending on the type of force, to 
reduce distortion between visual and fince outputs. Assistive 
forces and indexing features are also provided to achieve die 
enhanced cursor control. 
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MOUSE INTERFACE DEVICE AND METHOD 
FOR PROVIDING ENHANCED CURSOR CONTROL 

5 

BACKgHO OF THg INVEWTHON 

The present invention relates generally to interface devices for allowing humans to 
interface with computer systems, and more particularly to compute interface, devices that 
allow the user to provide input to computer systems and provide force feedback to the user. 

10 Computer systems are used extensively to implement many applications, such as word 

processing, data management, simulations, games, and other tasks. A computer system 
typically displays a visual, aivironment to a user on a display screen or other visual output 
device. Users can interact with the displayed environment to perform functions on the 
computer, play a game, exp^ence a simulated enviroiunent, use a computer aided design 

IS (CAD) system, etc. One visual environment that is particularly common is a graphical user 
interface (GUI). GUI*s present visual images which describe various gnqihical metaphors of a 
program or op^ting system implemented on the computer. Common GUI's include the 
Windows™ operating system from Microsoft Corporation and the MacOS operating system 
from Apple Computer, Inc. The user typically moves a displayed, user-controlled graphical 

20 object, such as a cvasox or pointer, across a computer screen and onto other displayed 
graphical objects or predefined screen regions, and then inputs a command to execute a given 
selection or operation. The objects or regions ("targets") can include, for example, icons, 
windows, pull-down menus, buttons, and scroll bars. Most GUI's are currently 2-dimaisional 
as displayed on a computer screen; however, three dimensional (3-D) GUI*s that present 

25 simulated 3-D enviroiunents on a 2-D screen can also be provided Other programs or 
environments that may provide user-controlled graphical objects such as a cursor or a view" 
controlled by the user include graphical ^web pages** or other environmrats offored on the 
World Wide Web of the Internet, CAD programs, video games, virtual reality, simulations, 
etc. 

30 The user interaction with and manipulation of the computer environment is achieved 

using any of a variety of types of human-computer interface devices that are connected to the 
computer systmi controlling the displayed environment In most s;^ems, die computer 
updates the enviroimient in response to the user's manipulation of a us^-manipulatable 
physical object ("user object") that is included in the interface device, such as a mouse. 
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joystick, etc. The computer provides feedback to the user utilizing the display screen and, 
typically, audio speakers. 

A computer mouse is a common user object used to interact with a GUI or other 
graphical environment A mouse (and other mouse-type devices such as a track ball) is 
5 typically used as a position control device in which displacement of the mouse in a planar 
workspace (e.g. on a mouse pad) is directly correlated to di^lacement of the user-controlled 
graphical object, such as a cursor, displayed on the screen. This displacement correlation may 
not be a one-to-one correspondence, since the cursor position may be scaled according to a 
constant mapping from the mouse position e.g., the mouse may be moved a distance of one 

10 indi on a mouse pad which causes the controlled cursor to move four inches across the screen. 
In most cases, small movements of the mouse are scaled to large motions of the cursor on ttie 
screen to allow the user to easily point to targets in all areas of the screen. The user can 
typically change the scaling or "pointer speed" of a cursor to a desired level, which is the 
ratio or scaling factor of cursor movement to mouse movement, using menus provided in the 

1 5 operating system or application program. 

The scaled cursor movement in a GUI works well for coarse cursor motion, which is 
the broad, sweeping motion of the cursor that brings the cursor from one global area on the 
screen to another. Acciuacy of cursor motion is not critical for coarse motion, but speed of 
the cursor is — ideally, the cursor traverses the desired distance on the screen quickly and 

20 efficiently. For such tasks, it is valuable for the cursor to move a large distance with small 
motions of the physical mouse hardware. However, a problem occurs in mouse-type devices 
when the user wishes to move the cursor a short distance or in small increments ("fine 
positioning"). For tasks in which accurate positioning of the cursor is needed, such as target 
acquisition tasks, the large scaling of mouse movement to cursor movement is inadequate or 

25 even harmful. For example, the user may wish to move the cursor onto a GUI target such as 
an icon or menu item. If very small motions of the mouse result in large cursor motion, the 
user may simply lack the manual dext^ty to acquire the target. Certain target acqiusition 
tasks where the targets are very small can be particularly challenging even if tihe ms^ping 
between the cursor and the mouse is reasonable for most other cursor motion ac^vities. For 

30 example, in drawing programs it is often required that a user position tiie cursor on a very 
small "point" or "node" on the screen; and in some cases, the target can be as small as a 
single display pixel For such situations, a scaling that causes large motions of the cursor for 
small motions of the mouse may make a target acquisition task physically impossible for the 
user. 

35 Mouse "ballistics" or ''ballistic tracking"' is typically used to alleviate the scaling 

problem for fine positioning of the cursor. Ballistics refo^ to the technique of varying the 

2 
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scaling between motion of a physical mouse and motion of a diq>layed cursor depending upon 
the velocity of the mouse in its workspace. The assumption is that if the user is moving the 
mouse very quickly, tiie user is likely performing a coarse motion^ task on the screen, and 
therefore the mouse driver scales small motions of the mouse to large motions of the cursor. 
5 Conversely, if the user is moving the mouse very slowly, the user is likely performing a fine 
positioning task on the screen, and the mouse driver scales small motions of the mouse to 
small motions of the cursor. Such a variable scaling technique is disclosed in U.S. Patent 
4,734,685 of Watanabe and 5,195,179 of Tokunaga. 

Many algorithms can be used for mouse ballistics. The simplest method is to 
10 designate a threshold velocity such that if the mouse is moving faster than the threshold 
velocity, a large scaling of cursor position is made so that small motions of the mouse cause 
large motions of the cursor, and if the mouse is moving slower flian the threshold velocity, a 
smaller scaling is made so that small motions of the mouse cause small motions of the cursor. 
A more sophisticated and more common method is to gradually change the scaling in 
15 accordance with mouse velocity using a continuous function. This can be a simple linear 
fimction, such as a direction relation between mouse speed and the distance the cursor moves 
for a given increment of mouse motion, or a non-linear function that is optimized in a 
particular way. The "mapping" of the cursor to the mouse is the method of translating the 
mouse position in its workspace to a cursor position on the display screen and may involve 
20 ballistics or other algorithms and scale factors. 

A problem occurs when standard ballistics techniques are used with force feedback 
interface devices. Force feedback interface devices allow a user to experience forces on the 
manipulated user object based on interactions and events within the displayed graphical 
enviroiunent. Typically, computer-controlled motors or other actuators are used to output 
25 forces on the user object in provided degrees of freedom to sunulate various sensations, such 
as an obstruction force when moving the cursor into a wall, a damping force to resist motion 
of the cursor, and a spring force to bias the cursor to move back toward a starting position of 
the spring. Force feedback devices can be implemented in many forms, such as a joystick, 
mouse, steering wheel, etc. 

30 When these and othCT types of forces are implemented in conjunction with mouse 

balUstics, a conflict occurs between the use of ballistics position and force feedback output 
In general, force feedback is generated based directly on motion of the mouse while visual 
feedback such as movement of the cursor does not correspond directly with motion of the 
mouse due to scaling and ballistics. As explained above, 1 mm displac^ent of the mouse 

35 may cause different visual results in cursor motion based on the mouse velocity when using 
ballistics. Thus, when implementing forces in an interface device, motion of ttie mouse can 
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no longer be consistently coirelated to cursor motion on the screen due to (he variable scaling. 
This is a particular problem for a force feedback mouse system because in general, feel 
sensations such as springs, surfaces, dampers, textures, masses, and other spatially related 
physical phenomenon rely on a constant, predictable mapping between the motion of the 
5 mechanical object (mouse), the forces generated on the mouse, and graphically displayed 
interactions on the screen (cursor motion). 

For example, a spring sensation can be used when the user manipulates a cursor to 
stretch a graphically displayed element on the screen, such as a line in a drawing program or a 
window in a GUI. The user positions the cursor on the line, and moves the cursor to stretch 

10 the line. Visually, this stretch is displayed based on cursor motion. In addition, the 
accompanying spring sensation outputs a resistance force that increases linearly with 
displacement. Traditional force feedback systems would use displacement of the moxise in its 
workspace as the displacement magnitude required to calculate the spring force. If no 
ballistics are in effect, no problem exists because the cursor displacement used in the visual 

15 display has a consistent, constant mapping to the mouse displacement used in the force 
"display." However, if ballistics are used to map physical mouse motion to displayed cursor 
motion, the motion of the cursor varies depending upon mouse velocity, causing a potoitial 
conflict: the stretch displayed visually is based on the variable mapping adjusted by the 
ballistic algorithm, while the stretch force is based on pure mouse motion. This conflict 

20 becomes a problem when, for example, the user stretches a line very quickly in one dkection 
from a starting position (both a mouse starting position and a screen starting position), 
changes to the opposite direction, and unstretches the line very slowly toward the starting 
position. The motion in the first direction has a large scaling of mouse motion to cursor 
motion, while the motion in the second direction has a small scaling of mouse motion to 

25 cursor motion. The user may move the mouse in its physical workspace exactly the same 
distance in both directions (returning to the mouse starting position), but on the screen the line 
might stretch very far (when moving fast), but then come back only a small distance (when 
moving slow). Thus, visually, the cursor did not return to the screen starting position. 

Using a traditional mouse, this frequently occurs and is not a problCTi* However, on a 
30 force feedback mouse where force display is based on mouse motion and visual display is 
based on cursor motion, a disconcerting dichotomy is noticed by the user. Since the feel of 
the stretch is based on mouse motion, if the user stretches in one direction and returns to the 
starting position, the user feels stretching the line a given displacement and then unstretching 
the line that same displacement, with an end result of no stretch. But, on the screen where the 
35 cursor mapping is based on velocity, the user would see the line stretching far in one direction 
and then unstretching only a small amount in ttie opposite direction so that the cursor is not 
yet back to the starting position. The user would visually expect to still feel some stretched 
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tension, but no sudi tendon exists since the mouse is back at the starting position in its ovm 
woikspace. Thus a problem is evident in force feedback mouse-type devices: ballistics are 
needed to allow dexterous cuisor control, yet ballistics distorts the seeing-feeling relationship. 

Thus, adjusting the mapping between physical mouse motion and di^layed cursor 
5 motion makes sense for graphical display, but does not make sense for force feedback where 
physical reahsm is critical to effective sensation generation. Therefore, for force feedback 
mouse systems, it would be prefenred to eliminate mouse ballistics. Unfortunately, such a 
force feedback mouse would not be optimized for both fine positioning and coarse motion, as 
is true of traditional mice. 

10 In addition, mouse ballistics causes another problem that causes difficulty in force 

feedback mouse implementation. As described above, moving the mouse in one direction 
quickly and then moving it back in the other direction slowly creates a situation where the 
mouse hardware has returned to its starting position but the cursor may be far away from its 
starting position. This illustrates that the frame of the cursor and the frame of the mouse have 

15 shifted or become offset If this offset becomes too large, the user may not be able to reach 
some parts of the screen within the range of motion of the mouse. In a typical mouse, the 
offset is corrected through a process called "indexing." Indexing is achieved in a typical 
mouse by lifting the mouse off the table and repositioning it after the mouse has hit a limit, 
while the cursor remains fixed in position. This brings the mouse and the cursor frames back 
• 20 ' to a smaller, more comfortable ofOset A force feedback mouse may have a limited workspace 
due to cost constraints and may not be able to be lifted off the table and r^ositioned. In 
addition, the mouse hitting a physical limit to its woikspace is disconcerting for a user 
expecting realistic force feedback. Thus, traditional indexing may not be practical. However, 
since ballistics needs indexing to restore the frame offsets, and since ballistics and indexing 

25 are both traditional mouse techniques that conflict with typical force feedback functionality, a 
solution is needed that reconciles both the ballistics and the indexing problom in force 
feedback hardware. 



5 
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STIMMARY OF THE INVENTION 



The presHit invention is directed to a force feedback interface which allows enhanced 
cursor control and does not compromise the fidelity of force feedback. Various embodimaits 
are presaited which distinguish when a conflict between cursor control and force feedback 
occurs and compensates for sudi conffict, as well as conq)ensating for any indexing problems 
occurring due to offsets in mouse and cursor movemait fiames. 

More specifically, a method of the present invention provides enhanced cursor control 
using a force feedback interface device coupled to host computer. The host computer displays 
a cursor within a graphical environment, such as a GUI, on a display device. A position of a 
user-grasped manipulandum, such as a mouse, in a device workspace is read as a referraice 
positiorL A cursor position is rqjoited to the host computer derived fix)m the referaice 
position, and the host computer displays the cursor within the graphical environment at a 
position corresponding to the cursor position. It is determined whether the cursor interacts 
with tiie grq»hical environment as to cause a force to be output on the manipulandum, and, if 
so, a force is output on the manipulandum. At least one of the reported cursor position and 
the output force allows the user of the force feedback interface device to fmely position the 
cursor witiiin the graphical environment and coarsely move the cursor in the gr^hical 
environment without causing a distortion in the output forces as expected to be experioiced 
by the user. 

A number of embodiments are particularly described. In some embodiments, tiie 
cursor position reported to tiie host computer is tiie reference position modified or scaled to 
allow enhanced cursor conttol. For example, the cursor position can be a ballistic position 
tiiat is ttie reference position modified by a ballistics algoritiim such tiiat cursor position is 
m^ped to manipulandum position based on a scaling derived &om a velocity of tiie 
manipulandum and allows enhanced cursor control. In ottier onbodiments, ofber types of 
variable scaling or variable mq)ping can be used, sudi as a pre<Uctivc scaling method that 
scales tiie cursor position based on wheflier a fine positioning mode is entered based on oflier 
criteria. In such ballistics or variable scaling/mapping embodiments, flie realism of output 
forces is maintained by determinmg mouse-based forces based on the reference data (position 
and motion of the mouse) rather tiwn flie cursor position (ballistic) data. Preferably, a k)cal 
microprocessor keeps track of botii reference data (local frame) and tiie ballistic data (display 
frame) and xises data fixjm each as appropriate. In one embodiment, tiie cursor position is a 
ballistic position except for when visual spring forces are output, which would cause a 
conflict in ttie cursor position and tiie experienced force. Thus, the cursor position sent to tiie 

6 
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host is the reference position modified by a constant mapping when the output force is a 
visual spring force instead of being modified by ballistics. In a different, preferred 
embodimmt, most motion-based output forces are again based on refi^rence data and the 
cursor position based on ballistic data, but particular forces that cause the conflict such as flie 
visual spring force are determined using the ballistic data. 

In other embodiments, enhanced cursor control is provided by outputting assistive 
forces. In one embodiment, a resistive force dedicated for enhanced cursor control, such as a 
damping for i&iction force, is output to slow movement of the manipulandum. The resistive 
force has a magnitude inversely based on a velocity of the manipulandum in the device 
workspace to allow the mouse to be slowed down for fine positioning tasks and freed or 
unencumbered for coarse positioning tasks. In a different embodiment, a detent force is 
associated with targets in the GUI for guiding the manipulandum to a particular position and 
thereby guiding said cursor to a corresponding position in the graphical environment In 
another embodiment, the detent forces are provided as a field of detents arranged in a 
predetermined spacing over a predetermined area surrounding the cursor, which are provided 
when the manipulandum is under a predetermined velocity and thus likely to need the detents 
for fine positioning of the cursor. In a different embodiment, obstruction forces simulating 
surfaces are arranged to assist the user to controlling the cursor in fine positioning tasks. 

An indexing feature of the present invention allows control over the cursor by the 
mouse whOT an offset exists between the position of the mouse in its workspace (local frame) 
and the position of the cursor on the display screen (display frame). The mouse device is 
coupled to a host computer that displays gr^hical objects in a graphical environment on a 
display screen and includes a mouse moveable in a mouse v^orkspace. The cursor is moved in 
a screen area based on the movement of said mouse. The mouse is determined whether it is 
within a predetermined distance to a physical limit of the mouse workspace, the 
predetermined distance being defined by a region next to said physical limit. A location of 
the mouse in the region is determined, and the location is used to provide control of 
movement of the cursor toward the screra edge conresponding to the physical limit. A cursor 
position is i^rted to the host con^>uter allowing control of the cursor to the edge of said 
screen area despite the ofifeet between local and display frames. A force feedback mouse 
device that provides an indexing function is similar to the method. 

In one embodiment of the indexing feature, the distance of penetration of the mouse 
into the region is sensed, and a resistive force is output on the mouse resisting the movemCTt 
mto the region. For example, the force rnay be a resistive spring force having a magnitude 
based on the distance of the mouse past the region border. The pmetration distance is used to 
provide control of movement of the cursor toward a screen limit of the display fi^e 

7 
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conesponding to the physical limit according to a isometric control paradigm. The speed of 
the cursor may be based on tfie distance of the mouse past the region border (e.g. the 
conq>ression of the simulated spring). In a different embodiment of an indexing feature, a 
first distance between a current position of the mouse and the closest phyacal limit of the 

5 workspace is determined* and a second distance between a current position of the cursor and 
the edge to the display screen corresponding to that physical limit is determined. A ratio 
between the first and second distances is used to determine a scaling for determining cursor 
position, thus allowing the cursor to be positioned to the edge of said screen area when (or 
before) the mouse reaches the physical limit to the workspace. In one embodiment, this 

10 scaling is performed only when the mouse is within a predetermined region adjacent to a 
physical limit of the mouse workspace. 

An interface device of the present invention providing enhanced cursor control over a 
cursor includes a user manipulatable physical object contacted by a user and movable m 
physical space, a sensor that detects movement of the physical object in physical space and an 
15 actuator that applies output forces on the physical object. A local microprocessor is 
preferably included for determining and reporting the cursor position to the host, determining 
and outputting forces, and determining indexing functions similarly as in the above 
embodiments. 

The methods and apparatus of the present invention advantageously provides 
20 enhanced control over a cursor in a graphical environment while not compromising the 
fidelity or CTqjected feel offeree feedback sensations based on motion of the mouse or other 
user object. This allows a user to perform fine positioning and coarse motion of the cursor as 
desired and still experience forces as expected based on interactions of the cursor in the 
graphical enviroimient. In addition, the indexing features of the present invention allow the 
25 user to control the cursor even when a large offset exists between the mouse and cursor 
positions in their respective fi^es, allows the met to reduce this offset, and substantially 
reduces the user's undesired expmeace of any hard, physical stops when the mouse reaches a 
physical limit 

These and other advantages of the presrat invention will become zppzxent to those 
30 skilled in the art upon a readmg of the foUowmg specification of the invention and a study of 
the several figures of the drawing. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a perspective view of one embodiment of a mouse interface system suitable 
for use with the present invention; 

5 Figure 2 is a perspective view of an embodiment of a mechanism suitable for the 

interface systan of Figure 1 ; 

Figures 3a-b are top plan views of the mechanism of Figure 2; 

Figure 4 is a block diagram of the system of Figure 1 for controlling a force feedback 
interface device of the present invention; 

10 Figure 5 is a diagrammatic illustration of the local fiame and host frame referenced in 

the present invention; 

Figure 6 is a flow diagram illustrating a first embodiment of a method of the present 
invention for providing enhanced cxusor control without compromising force feedback; 

Figures 7a-7f are diagranmiatic illustrations of the dichotomy between a display frame 
15 and a local frame when ou^utting a visual spring force; 

Figure 8 is a flow diagram illustrating a second embodiment of a method of the 
present invention for providing enhanced cursor control without compromising force 
feedback; 

Figure 9 is a flow diagram illustrating a third embodiment of a method of the present 
20 invention for providing enhanced asrsor control without compromising force feedback 
including a first embodiment of an indexing feature of the present invention; 

Figures lOa-lOc are diagrammatic illustrations demonstrating the isometric indexing 
function of the preset invention; 

Figure 11 is a flow diagram illustrating a fourth embodimOTt of a method of the 
25 present invOTtion for providing enhanced cursor control without compromising force 
feedback including a second cmbodhnent of an indexing feature of the present invention; 

Figure 12 is a flow diagram illustrating a fifth embodiment of a method of the present 
invention for providing enhanced cursor control without compromising force feedback; 
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Figure 13 is a flow diagram illustrating a sixth embodiment of a method of the present 
invention for providing enhanced cursor control without compromising force feedback; and 

Figure 14 is a flow diagram illustrating a seventh embodiment of a method of flic 
present invention for providing enhanced cursor control without compromising force 
feedback. 
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ny.TAII.ED pifigrRIPTION of PREFERB Fn KMBODIMENTS 



FIGURE 1 is a perspective view of a force feedback mouse interface system 10 of the 
present invention capable of providing input to a host computer based on the user's 
manipulation of the mouse and capable of providing force feedback to the user of the mouse 
system based on events occurring in a program implemented by the host computer. Mouse 
system 10 includes a mouse or "puck" 12, an interface 14, and a host computer 1 8. It should 
be noted that the term "mouse" as used herein, indicates an object 12 generally shaped to be 
grasped or contacted from above and moved within a substantially planar workspace (and 
additional degrees of freedom if available). Typically, a mouse is a smooth or angular shaped 
compact unit that snugly fits under a user's hand, fingers, and/or pahn, but may be shaped 
othwwise in other embodiments. 

Mouse 12 is an object that is preferably grasped or gripped and manipulated by a user. 
By "grasp," it is meant that users may releasably engage a portion of the object in some 
fashion, such as by hand, with their fingertips, etc. In the described embodiment, mouse 12 is 
shaped so that a user's fingers or hand may comfortably grasp the object and move it in the 
provided degrees of freedom in physical space; an example of a user's hand is shovra as 
dashed Une 16. For example, a user can move mouse 12 to correspondingly move a computer 
generated graphical object, such as a cursor or other image, in a graphical environment 
provided by computer 18. The available degrees of freedom in which mouse 12 can be 
moved are determined from the interface 14, described below. In addition, mouse 12 
preferably includes one or more buttons 15 to allow the user to provide additional conraiands 
to the computer system. 

It will be appreciated that a great number of other types of user manipulable objects 
("user objects" or "physical objects") can be used with the method and apparatus of the 
present invention in place of or in addition to mouse 12. For example, such objects may 
include a sphere sudi as attack ball, apuck. a joystick, cubical- or other-shaped hand grips, a 
receptacle for receiving a finger or a stylus, a flat planar surfece like a plastic card having a 
rubberized, contoured, and/or bumpy surfece, or other objects. 

Interface 14 interfaces mechanical and electrical input and output between the mouse 
12 and host computer 18 implementing the appUcation program, such as a GUI, simulation or 
game environment. Interface 14 provides multiple degrees of freedom to mouse 12; in the 
preferred embodiment, two planar degrees of freedom are provided to the mouse, as shown by 
arrows 22. In other embodiments, greater or fewer degrees of freedom can be provided, as 
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well as rotary degrees of fieedom. For many applications, mouse 12 need only be moved in a 
very small workspace area, shown as dashed line 24 in Figure 1 as an example. 

In a preferred embodiment, the user manipulates mouse 12 in a planar workspace and 
the position of mouse 12 is translated into a form suitable for intapretation by position 
sensors of the interface 14. The sensors track the movement of the mouse 12 in planar space 
and provide suitable electronic signals to an electronic portion of interface 14. The interface 
14 provides position information to host computer 18, which the host uses, for example, to 
display a cursor or other user-controlled graphical object In addition, host computer 18 
and/or interface 14 provide force feedback signals to actuators coupled to intoface 14, and the 
actuators generate forces on membm of the mechanical portion of the interface 14 to provide 
forces on mouse 12 in provided or desired degrees of fieedom. The user experiences the 
forces generated on the mouse 12 as realistic simulations of force sensations such as jolts, 
springs, textures, "barrief forces, and the like. 

The electronic portion of interface 14 may couple the mechanical portion of the 
interface to the host computer 18. The electronic portion is preferably included within the 
housing 26 of the interface 14 or, altematively, the electronic portion may be included in host 
computer 18 or as a separate unit with its own housing. More particularly, interface 14 
preferably includes a local microprocessor distinct and separate firom any microprocessors in 
the host computer 18 to control force feedback on mouse 12 independently of the host 
computer, as well as sensor and actuator interfaces. A suitable embodiment of the electrical 
portion of interface 14 is described in detail with reforence to Figure 4. 

The interface 14 can be coupled to flie computer 18 by a btis 17, which communicates 
signals between inter&ce 14 and computer 18 and also, in the preferred embodiment, provides 
power to the interfiice 14 (e.g. when bus 17 includes a USB interface). In other embodimCTits, 
signals can be sent between intaface 14 and compute: 18 by wireless transmission/rccq>tion. 
In preferred embodiments of the present invention, the intoface 14 serves as an iiq)Ut/output 
(I/O) device for the compute' 18. The interface 14 can also receive mpixts firom other input 
devices or controls ttiat are associated mouse syst«n 10 and can relay those inputs to 
computer 18. For example, commands sent by the user activating a button 15 on mouse 12 
can be relayed to con^uter 18 by intCTfece 14 to implraiCTt a command or cause the 
computer 18 to output a command to the interface 14. 

Host computer 18 is preferably a personal computer or workstation, such as an IBM- 
PC compatible computer or Macintosh personal computer, or a SUN or Silicon Graphics 
workstation. For example, the computer 18 can operate under the Wmdows™ or MS-DOS 
operating system in conformance with an IBM PC AT standard. Altematively, host computer 
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system 18 can be one of a variety of home video game systems commonly connected to a 
television set, such as systems available from Nintendo, Sega, or Sony. In oiher 
OTibodiments, home compute- system 18 can be a "set top box'* which can be used, for 
exan^le, to provide interactive television functions to users, or a "network-'* or "Intemet- 
conqput^' which allows users to interact with a local or global netwoik using standard 
connections and protocols such as used for the Internet and World Wide Web. Host computer 
preferably includes a host microprocessor, random access memory (RAM), read only memory 
(ROM), input/output (I/O) circuitry, and other components of compute well-known to those 
skilled in the art 

Host computer 18 preferably implements a host application program with which a user 
is interacting via mouse 12 and other periph^ls, if appropriate, and which can include force 
feedback functionality. For example, the host application program can be a simulation, video 
game, Web page or browser that implements HTML, VRML, or other instructions, scientific 
analysis program, virtual reality training program or application, or other application program 
that utilizes input of mouse 12 and outputs force feedback commands to the mouse 12. 
Herein, for simplicity, operating systems such as Windows™, MS-DOS, MacOS, Unix, etc. 
are also referred to as "application programs." In one preferred embodiment, an application 
program utilizes a graphical user interface (GUI) to present options to a user and receive input 
from the user. Herein, computer 18 may be referred as displaying "graphical objects" or 
"computer objects." These objects are not physical objects, but are logical software unit 
collections of data and/or procedures &at may be displayed as images by computer 18 on 
display screra 20, as is well known to those skilled in the art. A displayed cursor or a 
simulated cockpit of an aircraft might be considered a graphical object The host application 
program checks for input signals received from the electronics and sensors of interface 14, 
and ou^uts force values and/or commands to cause the output of forces on mouse 12. 
Suitable software drivers which interfece such simulation software with computer 
input/output (I/O) devices are available from Immersion Human Interface Corporation of San 
Jose, CaUfomia. 

Display device 20 can be included in host conq>uter 18 and can be a standard display 
screen (LCD, CRT, etc.), 3-D goggles, or any other visual output device. Typically, the host 
application provides images to be displayed on display device 20 and/or other feedback, such 
as auditory signals. For example, display screen 20 can display images from a GUI. Images 
describing a moving, first person point of view can be displayed, as in a virtual reality game. 
Or, images describing a third-person perspective of objects, backgrounds, etc. can be 
displayed. Alternatively, images from a simulation can be displayed. Images may be 
displayed and/or modified on display device 20 in response to user manipulations of mouse 
12. 

13 



wo 99/10872 



PCTA/S98/17445 



There are two primary "control paradigms*' of operation for mouse system 10: 
position control and rate control Position control is the more typical control paradigm for 
mouse and similar controllers, and icf&s to a mapping of mouse 12 in which displacement of 
the mouse in physical space directly dictates displacement of a graphical object The mapping 

5 can have an arbitrary scale factor, but the fundamental relation between mouse displacements 
and graphical object displacements should be presoit. Under a position control mapping, the 
computer object does not move unless the user object is in motion. Position control is a 
popular mapping for appHcations such as graphical user interfaces (GUI's) or medical 
procedure simulations. Position control force feedback roughly corresponds to forces which 

1 0 would be perceived directly by the user, i.e., they are " user-centric" forces. 

As shown in Figure 1, a "display frame" 28 is provided with the display screen 20 for 
defining the area of movement of a cursor in graphical environment. This frame can also be 
considered a "host frame*' , although the interface 14 may reference it as well. In contrast, the 
mouse 12 has a "local frame*' 30 allowed by the workspace in which the mouse 12 is moved. 
15 In a position control paradigm, the position (or change in position) of a user-controlled 
graphical object, such as a cursor, in display frame 30 corresponds to a position (or change in 
position) of the mouse 12 in the local frame 28. 

Rate control is also used as a control paradigm. This refers to a mapping in which the 
displacement of the mouse 12 along one or more provided degrees of freedom is abstractly 

20 mapped to motion of a computer-simulated object under control. There is not a direct 
physical mapping between physical object (mouse) motion and computer object motion. 
Thus, most rate control paradigms allow the user object can be held steady at a given position 
but the controlled computer object is in motion al a commanded or given velocity, in contrast 
to the position control paradigm that only allows the controlled computer object to be in 

25 motion if the user object is in motion. 

The mouse interface system 10 is useftil for both position control ("isotonic") tasks 
and rate control ("isometric") tasks. For example, as a traditional mouse, the position of 
mouse 12 in the workspace 24 can be directly mJ5)ped to a position of a cursor on display 
screen 20 in a position control paradigm. Alternatively, the displacement of mouse 12 in a 
30 particular direction against an opposing output force can command rate control tasks in an 
isometric mode, as described with refermce to (be indexing feature of Figure 13. 

Mouse 12 can be used, for example, to control a computer-generated gr^hical object 
sudi as a cursor displayed in a graphical computer environment, such as a GUI. The user can 
move the mouse in 2D planar workspace to move the cursor to grs^hical objects in the GUI or 
35 perform other tasks. In other graphical environments, such as a virtual reality video game, a 
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user can be controlling a computer play^ or vehicle in the virtual mvironment by 
manipulating the mouse 12. The conq)uter system tracks die position of the mouse with 
SGDSiOTS as the user moves it The computer system may also provide force feedback 
commands to the mouse, for example, when the user moves the graphical object against a 
5 generated surface such as an edge of a window, a virtual wall, etc. It thus appears and feels to 
the user that the mouse and the graphical object are contacting real surfaces. 

FIGURE 2 is a perspective view of one embodiment of mouse system 10 with the 
cover portion of housing 26 removed, showing the mechanical portion of interfece 14 for 
providing mechanical ii^ut and output m accordance with the present invention. Interface 

10 14 includes a mouse or other us^ manipulatable object 12, a mechanical linkage 40, and a 
transducer system 41. A base 42 is provided to support the mechanical linkage 40 and 
transducer system 41 on grounded surface 34. Mechanical linkage 40 provides support for 
mouse 12 and couples the mouse to a grounded surface 34, such as a tabletop or other support. 
Linkage 40 is, in the described embodiment, a 5-member (or "S-bar'*) linkage including a 

15 ground member 42, a first base member 44 coupled to ground member 42, a second base 
member 48 coupled to ground member 42, a link member 46 coupled to base member 44, and 
an object member 50 coupled to link member 46, base member 48 and to mouse 12. Fewer or 
greater numb^ of members in the linkage can be provided in alternate embodiments. 

Ground member 42 of the linkage 40 is a base for the support of the linkage and is 

20 coupled to or resting on a ground surface 34. The members of linkage 40 are rotatably 
coupled to one another through the use of rotatable pivots or bearing assemblies ("bearings'*) 
having one or more bearings. Base member 44 is rotatably coupled to ground member 42 by 
a grounded bearing 52 and can rotate about an axis A. Link member 46 is rotatably coupled 
to base mraiber 44 by bearing 54 and can rotate about a floating axis B, and base member 48 

25 is rotatably coupled to ground memb^ 42 by bearing 52 and can rotate about axis A. Object 
member 50 is rotatably coupled to base member 48 by bearing 56 and can rotate about 
floating axis C, and object member 50 is also rotatably coupled to link member 46 by bearing 
58 such that object membo- 50 and link member 46 may rotate relative to each o&er about 
floating axis D. Linkage 40 is formed as a five-member closed-loop chain arranged such that 

30 the members can rotate about then: respective axes to provide mouse 12 with two degrees of 
fireedom, i.e., mouse 12 can be moved within a planar workspace defined by the x-y plane, 
which is defined by the x- and y-axes as shown in Figure 2. Mouse 12 in the preferred 
embodiment is coupled to object member 50 by a rotary bearing 60 so that the mouse may 
rotate about floating axis E and allow the user some flexible movraient in the planar 

35 workspace. 
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Transducer system 41 is used to sense the position of mouse 12 in its woric^ace and 
to generate forces on the mouse 12. Transducer system 41 preferably includes sensors 62 and 
actuators 64. The soisois 62 collectively sense the movement of the mouse 12 in the 
provided degrees of freedom and s^d appropriate signals to the electronic portion of interface 
5 14. Sensor 62a senses mov^ent of link member 48 about axis A, and sensor 62b senses 
movement of base member 44 about axis A. These sensed positions about axis A allow the 
detmnination of the position of mouse 12 using known constants such as the lengths of the 
membm of linkage 40 and using well-known coordmate transformations. 

Sensors 62 are, in the described embodiment, grounded optical encoders that sense the 
10 intermittent blockage of an emitted beam. A grounded emitter portion 70 emits a beam which 
is detected across a gap by a grounded detector 72. A moving encoder disk or arc 74 is 
provided at the end of member 48 which blocks the beam in predetmnined spatial increments 
and allows a proc^sor to determine the position of the arc 74 and thus the member 48 by 
counting the spatial increments. Also, a velocity of member 48 based on the speed of passing 
1 5 CT.coder marks can also be determined. 

Transducer system 41 also preferably includes actuators 64 to transmit forces to 
mouse 12 in space, i.e., in two (or more) degrees of freedom of the user object. The housing 
of a grounded portion of actuator 64b is rigidly coupled to ground member 42 and a moving 
portion of actuator 64b (pref^xibly a coil) is integrated into the base member 44. The actuator 

20 transmits rotational forces to base member 44 about axis A. The housing of the grounded 
portion of actuator 64a is rigidly coupled to ground member 42 through the grounded housing 
of actuator 64b, and a moving portion (preferably a wire coil) of actuator 64a is integrated 
into base member 48. Actuator 64a transmits rotational forces to link member 48 about axis 
A, The combination of these rotational forces about axis A allows forces to be transmitted to 

25 mouse 12 in all directions in the planar workspace provided by linkage 40 through the 
rotational interaction of the members of linkage 40. 

In the preferred embodiment, actuators 64 are electromagnetic voice coil actuators 
v^ch provide force flirougb the interaction of a current in a magnetic field. The magnetic 
fields fix)m magnets of the actuators int^iact with a magnetic field produced 6om the wire coil 

30 when current is flowed in the coil, thereby producing forces on appropriate members. The 
magnitude or strragth of the force is dependent on the magnitude of the current that is applied 
to the coil, the number of loops in the coil, and the magnetic field strength of the magnets. 
The direction of the force depends on the direction of the current in the coil. Forces in the x- 
and y- directions of mouse 10 are thus produced. In other embodiments, other types of 

35 actuators can be used, both active and passive, such as DC motors, pneumatic motors, passive 
friction brakes, passive fluid-controlled brakes, etc. Voice coil actuators can also be used as 
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sensors to sense the velocity (and thus position and acceleration) of the members 44 and 48 
about axis A. 

In an alternate embodiment, the mechanism 14 can be used for a 3-D interface device 
that allows a user to move a user object 12 in three dimensions rather than the 2-D planar 
5 workspace disclosed. For example, in one embodiment, the entire mechanism 14 can be made 
to rotate about a grounded axis, such as axis H extending through the magnet assembUes 88.. 

As shown in FIGURE 3a, a workspace guide opening 76 is provided m ground 
membCT 42 to limit the movement of mouse 12 in the x*y plane and thus defines the physical 
workspace of the mouse 12. Guide opening 76 is a shallow opening in the ground member 42 

10 having sides which block movement of the mouse 12 beyond specified limits. A guide pin 78 
is coupled to the bearing 60 at axis E and extends down into the guide opening 76. Pin 78 
contacts one or more sides of the opening 76 when the mouse is moved to a limit in a 
particular direction. As shown, guide opening 76 has relatively small dimensions, allowing 
the mouse a workspace of approximately 0.9" by 0.9" in the described embodiment. This is 

15 typically adequate workspace for the user to move the mouse and control a graphical object 
such as a cursor on a display screen. In oth^ embodiments, differratly-sized guide openings 
can be provided for dififerently-sized workspaces, or other types of stops or guides can be used 
to prevent mov^ent past predetermined limits. The guide opening 76 is shown as square 
shaped, but it can be rectangular in other embodiments; for example, the dim^isions of 

'20 opening 76 can be made the same aspect ratio as the displayed area of display device 20. 
Figure 3a shows guide pin 78 approximately in the cento- of the guide opening 76. 

In FIGURE 3b, the mouse 12 (not shown) and axis E have been moved in the x-y 
plane of the workspace of the mouse. The movement of the mouse has been limited by the 
guide opening 76, whore guide pin 78 has engaged (he sidewall of the upper-left comer area 
25 of guide opening 76 and stops any fiutho: movement in the forward y-direction. 

FIGURE 4 is a block diagram illustrating the electronic portion of interface 14 and 
host computer 18 suitable for use with the presrat mvention. Mouse interface system 10 
includes a host computer 18, electronic interface 100, mechanical apparatus 102, and mouse 
or other user object 12. Electronic interface 100, mechanical ^aratus 102, and mouse 12 
30 can also collectively be considoed a force feedback interface device" 1 04 that is coupled to 
the host computer. 

As explained with reference to Figure 1, computer 18 is preferably a personal 

computer, workstation, video game console, or other computing or display device. Host 

computer system 18 commonly includes a host microprocessor 108, random access memory 

35 (RAM) 1 1 0, read-only memory (ROM) 1 1 2, input/output (I/O) electronics 1 14, a clock 1 1 6, a 
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display device 20, and an audio output device 118. Host microprocessor 108 can include a 
variety of available microprocessors from Intel, AMD, Motorola, or other manufacturers, and 
can be single microprocessor chip, or multiple primary and/or co-processors. Microprocessor 
108 preferably retrieves and stores instructions and other necessary data from RAM 1 10 and 

5 ROM 112 as is well known to those skilled in the art In the described embodiment, host 
computer system 18 can receive sensor data or a sensor signal via a bus 120 from sensors of 
system 10 and other information. Microprocessor 108 can receive data from bus 120 using 
I/O electronics 114, and can use I/O electronics to control other peripheral devices. Host 
computer system 18 can also output commands to interface device 104 via bus 120 to cause 

1 0 force feedback for the interface system 1 0. 

Clock 1 16 is a standard clock crystal or equivalent component used by host computer 
18 to provide timing to electrical signals used by host microprocessor 108 and other 
components of the computer system 18. Clock 1 16 is accessed by host computer 18 in the 
control process of the present invention to provide timing information that may be necessary 
15 in determining force or position, e.g., calculating a velocity or acceleration from position 
values. 

Display device 20 is described with reference to Figure 1. Audio output device 118, 
such as speakers, can be coupled to host microprocessor 108 via amplifiers, filters, and other 
circuitry well known to those skilled in the art. Host processor 108 outputs signals to 
20 speakers 118 to provide sound output to the user when an "audio event" occurs during the 
implementation of the host application program. Other types of peripherals can also be 
coxq>led to host processor 108, such as storage devices (hard disk drive, CD ROM drive, 
floppy disk drive, etc.), printers, and other input and output devices. 

Electronic interface 100 is coupled to host computer system 18 by a bi-directional bus 
25 120. The bi-directional bus sends signals in either direction between host computer system 18 
and the interface device 104. Bus 120 can be a serial interface bus providing data according 
. to a serial communication protocol, a parallel bus using a parallel protocol, or other types of 
buses. An interface port of host computer system 18, such as an RS232 serial interface port, 
connects bus 120 to host computer system 18. In another embodiment, an additional bus 122 
30 can be included to communicate between host computer system 18 and interface device 13. 
Bus 122 can be coupled to a second port of the host computer system, such as a game port"' , 
sudi that two buses 120 and 122 are used simultaneously to provide an increased data 
bandwidth. One preferred serial interface bus used in the present invention is the Univ^sal 
Serial Bus (USB). The USB standard provides a relatively high speed serial interface that can 
35 provide force feedback signals in the present invoition with a high degree of realism. USB 
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can also source power to drive actuators 64 and oth^ devices of the present invention, and 
can provide timing data tliat is encoded along with differential data. 

Electronic interface 100 includes a local microprocessor 130, local clock 132, local 
memory 134, sensor interface 136, and actuator interface 138. Interface 100 may also include 
5 additional electronic components for communicating via standard protocols on buses 120 and 
122, In various embodiments, electronic int^ace 100 can be included in mechanical 
apparatus 102, in host computer 1 8, or in its own separate housing. 

Local microprocessor 130 preferably coupled to bus 120 and may be closely linked to 
mechanical apparatus 102 to allow quick communication with other components of the 

10 interface device. Processor 130 is considered "local" to interface device 104, where "local" 
herein refers to processor 130 being a separate microprocessor from any processors 108 in 
host computer 18, "Local" also preferably refers to processor 130 being dedicated to force 
feedback and sensor I/O of the interface system 10, and being closely coupled to sensors and 
actuators of the mechanical sq)paratus 102, such as within the housing of or in a housing 

15 coupled closely to ^paratus 102. Microprocessor 130 can be provided with software 
instructions to wait for commands or requests from computer host 18, parse/decode the 
command or request, and handle/control input and output signals according to the command 
or request. In addition, processor 130 preferably operates independently of host computer 18 
by reading sensor signals and calculating appropriate forces from those sensor signals, time 

20 signals, and force processes selected in accordance with a host command, and output 
appropriate control signals to the actuators. Suitable microprocessors for use as local 
microprocessor 200 include the MC68HC711E9 by Motorola, the PIC16C74 by Microchip, 
and the 80930 from Intel, for example. Microprocessor 130 can include one microprocessor 
chip, or multiple processors and/or co-processor chips, and/or digital signal processor (DSP) 

25 fimctionality. 

For example, in one host-controlled embodiment that utilizes microprocessor 130, 
host computer 18 can provide low-level force commands over bus 120, which microprocessor 
130 directly transmits to the actuators. In a different local control onbodiment, host computer 
system 18 provides high level supervisory commands to microprocessor 130 over bus 120, 

30 and microprocessor 130 maiuiges low level force control loops to sensors and actuators in 
accordance with the high level commands and independently of the host computer 18. In the 
local control embodiment, the microprocessor 130 can process ]iq)utted sensor signals to 
determine appropriate output actuator signals by following the instructions of a force 
process'* that may be stored in local memory and includes calculation instructions, formulas, 

35 force magnitudes, or other data. The force process can command distinct force s^isations, 
such as vibrations, textures, jolts, or even simulated interactions between displayed objects. 
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For example, a rigid surface is generated on compute screen 20 and a computer object 
(e.g., cursor) controlled by the user collides with the surface. In a prefmed embodiment, 
high-level host commands can be used to provide the various forces associated with the rigid 
sur&ce. A local control mode using microprocessor 130 can be helpful in increasing the 

S response time for forces applied to the user object, which is essential in creating realistic and 
accurate force feedback. For example, it is preferable that host computer 18 send a ^'spatial 
rqjresentation" to microprocessor 130, which is data describing the locations of some or all 
the graphical objects displayed in a GUI or other gr^hical environment which are associated 
with forces and the types/characteristics of these graphical objects. The microprocessor can 

10 store such a spatial rq)resentation in memory 134, and thus will be able to determine 
inta*actions between the user object and graphical objects (such as the rigid surface) 
indq)endently of the host computer. In addition, the microprocessor 130 can be provided 
with the necessary instructions or data to check sensor readings, detennine cursor and target 
positions, and detennine output forces independently of host computer 18. The host can 

IS implement program fimctions (such as displaying images) when s^propriate, and 
synchronization commands can be communicated betwera processor 130 and host 18 to 
correlate (he microprocessor and host processes. Also, memory 134 can store predetermined 
force sensations for microprocessor 130 that are to be associated with particular types of 
graphical objects. Alternatively, the computer 18 can directly send force feedback signals to 

20 the interface 14 to generate forces on mouse 12. 

Sensor signals used by microprocessor 130 are also reported to host computer system 
18, which updates a host application program and ou^uts force control signals as appropriate. 
For example, if the user moves mouse 12, the con4>uter system 18 receives cmrsor position 
signals indicating this movemrat and can move a displayed cursor in response. In an alternate 
25 embodiment, no local microprocessor 130 is included in interface system 10, and host 
computer 18 directly controls and processes all signals to and fix)m the interface 100 and 
mechanical inter&ce 102. 

A local clock 132 can be coipled to fbe microprocessor 130 to provide timing data, 
similar to system clock 116 of host computer 18; the timing data tnight be required, for 
30 exanq)le, to compute forces ou^ut by actuators 64 (e.g., forces dq)endmt on calculated 
velocities or oQier time dependent factors). In alternate embodiments using the USB 
communication int^&ce, timing data for microprocessor 130 can be retrieved fix>m the USB 
interface. 

Local memory 134, such as RAM and/or ROM, is pref<^:ably coupled to 
35 microprocessor 130 in interface 100 to store instructions for microprocessor 130 and store 
temporary and ofber data. Microprocessor 130 may also store calibration parameters in a 
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local memoiy 134 sudi as an EEPROM hkmory 134 may also be used to store the state of 
the force feedback device, including a refa:ence position, cnnent control mode or 
configuration, etc. 

Sensor interface 136 may optionally be included in electronic int^ace 100 convert 
5 sensor signals to signals that can be interpreted by the microprocessor 130 and/or host 
computer system 18. For example, sensor interface 136 can receive signals from a digital 
sensor such as an encoder and convert the signals into a digital binary number rqsres^ting 
the position of a member or component of mechanical apparatus 14. An analog to digital 
converts (ADQ in sensor inter&ce 136 can convert a recdved analog signal to a digital 
10 signal for microprocessor 130 and/or host computer 18. Alternately, microprocessor 130 can 
perform these interface functions without the need for a sq)arate saisor mtaface 136. Or, 
sensor signals from the sensors can be provided directly to host coniputer syst^ 18. Other 
types of interface circuitry 1 36 can also be used. 

Actuator interface 138 can be optionally connected between the actuators 64 and 
15 microprocessor 130. Inter£EK:e 138 converts signals from microprocessor 130 into signals 
appropriate to drive the actuators. Interface 138 can include pow^ amplifiers, switches, 
digital to analog controllers (DACs), and other componrats. Such interfaces are well known 
to those skilled in the art. In alternate embodiments, interface 138 drcuitry can be provided 
within microprocessor 130 or in the actuators. 

20 In the described embodiment, powo: is supplied to the actuators 64 and any other 

components (as required) by the USB. Alternatively, power Scorn the USB can be stored and 
regulated by interface 100 or q)paratus 102 and thus used when needed to drive actuators 64. 
Altonatively, a power siq)ply 140 can optionally be coupled to actuator interface 138 and/or 
actuators 64 to provide electrical power. 

25 Mechanical apparatus 102 is coupled to electronic interface 100 preferably includes 

sensors 62, actuators 64, and linkage 40, Sensors 62 sense the position, motion, and/or other 
characteristics of mouse 12 along one or more degrees of freedom and provide signals to 
microprocessor 130 including information rq>resentative of those characteristics. Typically, a 
sensor 62 is provided for each degree of freedom along whidi mouse 12 can be moved, or, a 

30 single con^und sensor can be used for multiple degrees of freedom. Example of sensors 
suitable for embodiments described herein are rotary or linear optical encode, 
potentiometers, non-contact sensors (e.g.. Hall effect magnetic sensors, optical sensors, lateral 
effect photo diodes), velocity sensors (e.g., tachometoi), or acceleration smsors (e.g., 
accelerometers). Furthermore, eitiier relative or absolute sensors can be employed. 
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Actuators 64 transmit forces to mouse 12 in one or more directions along one or more 
degrees of fieedom in response to signals output by microprocessor 130 and/or host computer 
18. i.e., they are "computer controlled." Typically, an actuator 64 is provided for each degree 
of freedom along >vhich forces are desired to be transmitted. Actuators 64 can include two 

5 types: active actuators and passive actuators. Active actuators include linear current control 
motors, stepper motors, pneumatic/hydraulic active actuators, a torquer (motor with limited 
angular range), a voice coil actuator, and other types of actuators that transmit a force to an 
object. Passive actuators can also be used for actuators 64, such as magnetic particle brakes, 
friction brakes, pneumatic/hydraulic passive actuators, or passive damper elements and 

10 generate a damping resistance or friction in a degree of motion. In some embodiments, all or 
some of sensors 62 and actuators 64 can be included together as a sensor/actuator pair 
transducer. 

Mechanism 40 is preferably the five-member linkage 40 described above, but can also 
be one of several types of mechanisms. Mouse 12 can alternatively be a puck, joystick, or 
1 5 other device or article coupled to linkage 40, as described above. 

Other input devices 141 can optionally be included in system 10 and send input 
signals to microprocessor 130 and/or host computer 18. Such input devices can include 
buttons, such as buttons IS on mouse 12, used to supplement the input from the user to a GUI, 
game, simulation, etc. Also, dials, switches, sensors, voice recognition hardware (with 
20 software implem^ted by host 1 8), or other mpnl mechanisms can be used. 

Safety or "deadman" switch 150 is preferably included in interface device to provide 
a mechanism to allow a usct to override and deactivate actuators 64, or require a user to 
activate actuators 64, for safety reasons. Safety switch ISO is coupled to actuators 64 such 
that the user must continually activate or close safety switch ISO during mianipulation of 

2S mouse 12 to activate the actuators 64. If, at any time, the safety switch is deactivated 
(opened), power is cut to actuators 64 (or the actuators are otherwise deactivated) as long as 
the safety switch is opened. Safety switch 150 can be a mechanical or optical switch located 
on mouse 12 or on a convenimt surface of a housing 26, an electrostatic contact switch to 
sense contact of the user, or a hand-weigjit safety switch. The state of the safety switch can be 

30 sent to the microprocessor 130 and/or to host 18. 

In some embodimrats of mterface system 10, multiple mechanical apparatuses 102 
and/or electronic interfaces 100 can be coupled to a single host computer system 18 ihrougfh 
bus 120 (or multiple buses 120) so that multiple users can simultaneously interface with the 
host application program (in a multi-playa: game or simulation, for example). In addition, 
35 multiple players can interact in the host explication program with multiple intedacc systems 
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10 using netwoiked host computers 18, as is well known to those skilled in the art. Also, the 
interface device 104 can be coupled to multiple host computers; for example, a local host 
computer can display images based on data received 6om a remote host computo' coupled to 
the local host through a network. 



Enhanced Cursor Control and Force Feedback 

One aspect of the present invention is concerned with mouse system 10 allowing an 
oihanced degree of control over a cursor for a user, as well as allowing high-fidelity force 
feedback that is not compromised by the control allowed over the cursor. The enhanced 
degree of cursor control includes fine positioning of the cursor for target acquiring and other 
tasks, as well as coarse positioning of the cursor that is unencumbered and uninhibitied by the 
fine positioning. Anotha* aspect of the present invention is to allow control ova: the cursor 
without limits to physical movement of the mouse (or other object) to become intrusive to the 
user, i.e., the device incorporates an "indexing" feature that corresponds to the case in a non- 
force-feedback mouse of the user repositioning the mouse in its workspace to reduce the 
offset between the mouse frame and the host computer frame. There are several different 
embodiments described herein that include these features. Although the term "mouse" is 
used in the following embodiments, it is intended that other types of interface devices and 
user object may also be used with the present invention. In addition, the various 
embodiments presented below are described for use with the preferred local microprocessor 
130 (or other dedicated processing circuitry on the interface device 104); howeva:, a host 
compute 18 can implement the embodiments of the present invention (with any appropriate 
modifications) if no local microprocessor is present in a particular hardware embodimoit. 
Alternatively, the host compute can implement some functions (such as ballistics calculations 
and indexing calculations) while the microprocessor implements other functions. It is 
assumed in the methods below that host computer 18 is displaying a graphical environment 
such as a GUI, game, simulation, etc. on display device 20. 

The methods described below may be implemented witii program instructions or code 
stored on or transferred through a computer readable medium. Such a computer readable 
medium may be digital memory chips or other memory devices; magnetic media such as hard 
disk, floppy disk, or tape; or otiier media such as CD-ROM, DVD, PCMCIA cards, etc. The 
computer readable medium may be included in the interface device 104, in host computer 18, 
or in both. The program instructions may also be transmitted through a chaimel to inter&ce 
device 14 from a dififi^ent source. 
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FIGURE 5 is a diagrammatic illustFation of the local frame 30 and display fi^me 28 
and their relationship. The local frame 30 is provided in the available workspace iii which the 
mouse or other user object may be moved^ In the embodiment described with reference to 

Figure 2, for example, the dimensions of the local frame 30 are defined by guide opening 76 
5 in the base 42, which may be approximately 1" X T'. Physical Umits to the local frame 30 
are provided by guide pin 78 physically impacting a wall of opening 76. The mouse 
woricspace may be defined and limited by other mechanisms or structures in other 
embodiments. 

Display firame 28 is shown as a rectangle overlapping the local frame 30. Display 
10 frame 28 is the visible^ displayed area on display device 20, such as the displayed portion of a 
video screen, on which a user controlled graphical object, such as cursor 180, may be moved. 
In Figure 5, the display firame 28 is shown as the same size as local frame 30 to emphasize 
certain concepts in the present invention. However, in actuality, the display fi:ame 28 is 
typically larger in actual size than the local fi:ame; for example, a computer monitor may have 
15 a screen of 15" X 11" compared to the local fi^e dimensions 1" X 1". Thus, movement in 
local firame 30 is scaled up to allow movement across the entire area of display fi-ame 28. 

Local frame 30 has a local origin 182 Scorn which x and y coordinates of the mouse 
device in its workspace are referenced. Cursor 180 is shown in Fig. 5 to represent the position 
of both the cursor 180 displayed in display fi^e 28 as well as the current position of the 

20 mouse 12 in the local fi:ame 30 (e.g., the position of axis E and guide pin 78 in the 
embodiment of Fig. 2), where the tip of the cursor indicates the precise position. The guide 
pin 78 (shown as the tip of cursor 180) thus has a position of (X^local, Y_local) in the 
example of Figure 5. Likewise, display fiame 28 has a screen origin 184 from which x and y 
coordinates of the cursor 180 displayed on the screen 20 are referenced. The cursor 180 thus 

25 has a position of (X_screen, Y_screen) in the example of Figure 5 . 

Jn Figure 5, the display firame 28 is shown offset &om local frame 30. This has 
implications for the indexing feature of the present invention, which is described in greater 
detail below. 

FIGURE 6 is a flow diagram illustrating a first method 200 of the present invmtion 
30 for implCTienting oihanced cursor control and realistic force feedback in mouse system 10. 
In method 200, ballistics are provided to allow fine positioning and coarse motion of the 
cursor, but reference data is used for determination of position-based forces to provide more 
realistic force feedback. In the preferred embodiment, the local microprocessor 130 
determines the ballistic positions of the cursor, and thus allows the force sensations to be 
35 calculated based on reference data. 
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The method begins at 202. In step 204, the mouse position in the loeal frame 30 (Le., 
mouse woikspace) is read by the local microprocessor 130 and is considered tiie ^'reference 
position." For exan^le, the position can be described as x and y coordinates referaiced to 
workspace origin 182. This position is preferably stored in a storage area such as local 
5 memoiy 134 and is kept available for retrieval by the microprocessor at a later time. 

In step 206, a ballistic position is determined from reference data. "Reference data", 
as refOTed to herein, is data describing the position and/or motion of the mouse in local frame 
30^ For sample, the reference position obtained in step 204 is reference data, and the 
velocity and acceleration of the mouse in local frame 30 is also reference data In the 

10 described embodiment, the ballistic position is determined using standard ballistic algorithms 
and methods based on the velocity of the mouse. For example, the current velocity of the 
mouse in the local fi^e 30 can be determined by examining stored positions of the mouse 
ova: time read by the sensors of the interface device, or by examining timing signals or pulses 
from sensors, or from differentiating an analog signal, etc. Timing data can be obtained, for 

15 example, using local clock 132. One preferred embodiment uses a haptic accelerator on the 
interface device 104 to determine velocity and/or acceleration of the mouse 12 in its local 
fitime 30 and to input such reference data to the local microprocessor 130. 

Once the velocity of the mouse is known, the local microprocessor can use a ballistics 
algorithm to determine how to map the position of the cursor to the mouse. A "ballistic scale 

20 factor*' (BSF) can be determined based on the mouse velocity, where the BSF is the ratio 
between movement of mouse in its workspace and movement of the cursor on the screen. The 
BSF is determined such that low mouse velocities create small cursor motions (lower value 
BSF), and large mouse velocities create large, fast cursor motions (higher value BSF). The 
BSF can be determined based on a continuous function to determine the precise scaling, or a 

25 simpler discrete function can be used in which one or more velocity thresholds are checked to 
determine the value of the BSF. A diange in position of the cursor is then detennined as ttie 
BSF multiplied by fho change in reference position (as determined using values from step 
206). The ballistic position is prefoably calculated as the old cursor position (fiie position of 
the cursor in display frame 28 ui the last iteration of mediod 200) plus the change in position 

30 of the cursor just detennined. Thus, if the mouse is traveling slowly, (hen the ballistic 
position is scaled down or not scaled at all since the user probably is poforming fine 
positioning of the cursor and would like to move the cursor in small increments. If the 
velocity is large, the ballistic position is scaled higher since the user probably is performing 
coarse movement to get the cursor across the screen quickly. Such ballistics scaling is well 

35 known to those skilled in the art. One or more ballistic positions can be referred to as 
"ballistic data", and the ballistic positions are preferably stored by the microprocessor in 
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local memory as the display fiame data. Thus, the microprocessor keq)S track of both local 
frame data (reference data) and display frame data (ballistic data). 

Alternatively, other methods can be used besides ballistics to vary the scaling or the 
mapping of the cursor position to allow fine positioning and coarse motion of the cursor. For 

5 example, a predictive type of scaHng of the present invention can be used, which is more 
"friendly" to force feedback implenientations than the standard ballistics of the prior art. 
Such predictive scaling only implements a fine-positioning scaling that is different from a 
coarse-movement scaling whra it is deemed necessary for greater control over the cursor. 
That is, other criteria besides mouse velocity are used to detomine when to alter the scaling 

10 of the cursor position from the mouse position. For example, the local microprocessor can 
examine positions of the mouse (or the cursor) over a predetermined p^od of time to see if a 
fine positioning mode is entered. The microprocessor checks whether the cursor has moved 
completely within a small region of predefined size for longer than a predetmnined period of 
time. Hie region can be defined by a radius or rectangular area surrounding the cursor, for 

15 example, a region having a radius of (fraction of screen size) can be used. The predetermined 
period of time is some time period long enough to indicate that the user is attempting to 
acquire a target or perform some other fine positioning task and may be having some 
difficulty; for example, 3 seconds can be used, or the time may depend on the particular task. 
In addition, the cursor should be in motion, since if the cursor is still, then the user may 

20 simply have taken his or her hand off the mouse, and fine positioning mode should not be 
entered. 

If such conditions apply, then it is assumed/predicted that the user needs the assistance 
of fine positioning mode to perform the desired task, and the cursor position is set to an 
adjusted or scaled reference position that has been scaled for fine positioning. For example, 

25 the cursor position can be scaled to 1/4 the mouse position so that 4 times the mouse motion 
is required to achieve an equivalent cursor motion. This allows longer mouse movemrats to 
move the cursor in shorter ina:ements and greatly assists fine positioning of the cursor. 
Alternatively, a ballistic algorithm can be employed in step 364 which makes cursor motion 
based on the velocity of the mouse. Presumably the usor is moving the mouse slowly so that 

30 the cursor motion is scaled down according to the ballistics algorithm. If the conditions do 
not Bpply, the cursor position can be scaled according to a constant coarse mapping, since the 
cursor was not deemed to have motion sufficient to change the scaling of the cursor for fine 
positioning. 

In addition, the microprocessor can check for conditions to exit the fine positioning 
35 mode of the cursor. For example, the user may press button 15 on mouse 12 (or otiier input 
devices) to manually command the mouse to exit fine positioning mode. Or, if the cursor is 
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outside the small predefined region (which does not move with the cursor or mouse once fine 
positioning mode is enta:ed) then fine positioning mode is exited. Or, if the mouse has 
remained still and unmoving for a minimum predetermined period of time in the region, fine 
positioning mode can be exited. Thus, other conditions besides mouse velocity det^mine 

5 when different scaling is q)plied to the cursor position. This method is useful for force 
feedback devices because it restricts the variable scaling of cursor position to only cases when 
the cursor is moving within a very small region and is close to converging upon a target. 
Because only small motions are involved, the dichotomy between force feedback sensations 
and the visual motion is not as noticeable to the user. Nevertheless, the force feedback is still 

10 preferably modified to minimize any distortion, as described below. 

In stop 208, the local microprocessor determines interactions between the ciusor and 
the GUI (or other graphical environment, such as a 3'D envirorunent) using the cursor 
position. The cursor 180 displayed by the host may have encountered or contacted a 
graphical object or region in the GUI that is associated with a force sensation, so that the force 

15 sensation should be output on the mouse. For example, the user may have moved the cursor 
onto an icon and held down a button 15 on the mouse to drag the icon, which might cause a 
damping or inertia force to be output when the icon is dragged. Or, the cursor may have been 
moved over a window border which causes a spring or detent force to be output on the mouse 
to indicate the location of the border. A different situation in which a force may be output is 

20 when an event has taken place in the GUI or other graphical environment which causes a 
force soisation on the mouse. For example, a sound may be output indicating a mail message 
has been received, which in tum causes an attractive force on the mouse toward a mail 
program icon. In some cases, multiple force sensations are output that are overlaid on each 
other. In the preferred embodiment, the local microprocessor 130 is commanded with high 

25 level host commands bom the host computer 18 to implement one or more local processes 
diat locally check mouse positions and other conditions and output forces whai particular 
interactions or events occur in the graphical envirorunent For example, as explained above, 
the local microprocessor can previously be sent a layout of graphical objects in the GUI fix>m 
the host computer to allow the microprocessor to check for collisions or interactions. 

30 Alternatively, the host computer can check these collisions and can send a host command to 
cause the local processor to immediately ouQ)ut a force wh^ the host computer determines 
fliat such a force is ^ropriate. 

It is important to note that the interaction of the cursor with other objects in the 

display fczme graphical environment is generally determined based on the ballistic data &om 

35 step 206. That is, the cursor position in the gr£q)hical environment has been scaled according 

to mouse velocity and thus may be different than the position of the mouse in the local firame 

30. When the local micropix)cessor determines when interactions of the cursor and graphical 
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objects occur in stq> 208 (as is preferred), the local miaxrprocessor thus uses the cursor 
position in the display frame 28 to determine the location of the cursor and any int^actions of 
&e cursor in the display frame. 

The process continues to step 210, where independent forces are calculated 
5 "Independent forces", as referred to herein, are those forces which are not based on position 
or motion of the mouse or cursor and thus require neither reference data nor ballistic data to 
be calculated or gmeraled. For example, time-based or periodic forces, such as a vibration or 
a jolt, are simply output on the usct object at predetennined and/or repeating time intervals 
and durations, in specified directions, at a specified starting time, and at predetermined 
10 magnitudes. To calcul^tte these forces, in contrast with the damping and inorda forces 
described above, neither the position, velocity, or acceleration of the mouse is required. Thus, 
these forces can be calculated normally with no need to retrieve reference data or ballistic 
data. 

In step 212, remaining forces are calculated using the refemice data. The remaining 
IS forces are forces based on a position, velocity, and/or acceleration of the mouse 12. For 
example, the calculation of a damping force is generally performed using the relation F - Bv, 
where v is the velocity of the object, B is a damping constant, and F is the resulting daztq>ing 
force. Here, v is preferably based on the velocity of the mouse, so the reference data is used 
to determine v. Similarly, an inertia force uses acceleration in its calculation, which is based 
20 on the reference data in step 214. Likewise, a spring force is typically modelled using F = kx, 
whCTe X is the displacement of the object and k is a spring constant. The displacemOTt x is of 
the mouse and is thus provided using the reference data. Also, a fiiction force can be 
modelled as F = f * (v/ |v|), whare f is a fiiction constant, v is the displacement of the object, 
provided as reference data, and v / |v| is used to indicate the opposite direction to the velocity 
25 of the mouse (since fiiction opposes motion). Friction forces can be determined in other ways 
as well. 

The use of reference data to calculate such position/motion based forces is one of the 
features of the present invention. To effectively reconcile the use of balUstic data for cursor 
positioning with the ou^ut of force feedback, the present invration uses reference data in the 

30 calculation of forces while providing ballistic data to the host conq)uter to control the position 
of the cursor on the screen. This allows forces to be realistically based on the actual position 
of the mouse in its local fiame, yet also allows the cursor to based on ballistic data to allow 
fine positioning and coarse movement of the cursor in the display firame. If both the cursor 
and the forces were based on reference data, then tiie advantage of more control oyer cursor 

35 motion gained by the use of ballistic data would be lost. Thus, the present inventiori 
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advantageously provides ballistic cursor positioning while providing forces based on 
referCTce data. 

The local microprocessor 130 allows this use of two sets of data to be easily 
implemented, since the microprocessor can keep track of the reference local frame 30 and the 
S ballistic display frame 28 separately and can choose data from each set as needed. Thus, the 
microprocessor can select ballistic data when determining the location of the cursor in relation 
to other graphical objects in the GUI, can select reference data when calculating forces, and 
can select ballistic data to report to the host computer (described below). The use of local 
microprocessor 130 for local force generation allows this implementatioa 

10 For example, a texture force can make use of both frames of data. In one example, a 

texture can be a pulsating force based on the position of the user object, as if dragging the user 
object over bumps or a grating having a particular spacing. One way to implement a texture 
force sensation is to ou^ut a damping force that is modulated based on mouse position, i.e., 
the damping is tumed on and off in sequence to simulate bumps at a given spatial frequency. 

IS Using the two stored frames of data, the damping resistance can be generated using the 
reference data since damping depends on the velocity of mouse motion, and the spatial 
modulation of the damping force over the textured region can be based on ballistics data since 
the position of the cursor on the screen dictates when a bump would be felt. 

Another solution of the present invention is to use the ballistic data to both position 

20 the cursor and to generate all forces. This is a simpler solution in that, once ballistic data is 
determined from the reference data, the refsreacc data is no longer needed and the ballistic 
data can be used for all purposes. However, such a solution is not the preferred embodim^t. 
This is because, while the ballistic data would allow fine positioning and coarse motion of the 
cxirsor, generation of many types of force sensations with ballistic data would distort or 

25 diminish the realism of those force sensations. For example, damping force sensations are 
typically based on velocity, such as F = Bv. Damping is often used to simulate the viscosity 
of a inaterial, such as the feel of moving through a Uquid If the ballistic velocity of die 
cursor is used for v, then the viscosity of a liquid would vary d^ending on whether the mouse 
is moved slow or frist as governed by the ballistics algoiitimL The ballistic velocity is a sc^ 

30 velocity, different from the actual velocity of the mouse, ttiat would cause a much difierrat 
magnitude of damping than the user expects to feel by moving the mouse. A similar situation 
occurs for an inertia force diat is based on the acceleration of an object. If the ballistic data is 
used to calculate acceleration, a different inertia will be felt than the inertia that the user 
expects by moving the mouse. The inertia force would feel as if the moving object were 

35 heavier when it was moved faster, and lighter when moved slower, which is typically 
undesired. Thus, using reference data for the determination of such forces as in method 200 
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provides more realistic forces for the met since tlie reference data describes actual 
position/motion of the mouse in local firame 30. The force designer may not want springs to 
change their perceived stif&ess, dampers to change their perceived viscosity, and inertias to 
change their perceived mass as the mapping shifts using a ballistic algori&m, such that using 
5 the ballistic data in force determination is undesired. 

In step 214 the forces determined in steps 210 and 212 (as well as any other forces 
determined for other reasons) are summed and output in step 214. This step may be 
p^ormed at any time after step 212, or concurrently with remaining steps 216-222. The total 
force is output by the interface device on the mouse grasped by the user. Actuators 64 are 
10 preferably controlled by the local microprocessor 130 to output this force. The output force 
may also have a specified duration, direction, frequency, and other parameters to which the 
local microprocessor conforms the output force. The local microprocessor knows these 
parameters by retrieving them as standard or stored parameters, or may receive new 
parameters directly from the host computer 1 8. 

15 In next step 216, the process checks whether a visual spring force was calculated in 

step 212 (or determined to be required in step 208). A visual spring force is special in that, 
when reference data is used to determine forces and ballistic data is used to position the cursor 
on the screen, an undesired dichotomy between the displayed spring and the feel of the spring 
results. A "visual spring" force is to be distinguished from a "clipped spring" force. The 

20 visual spring allows tiie user to see the cursor moving on the screen following the e?q)ansion 
or contraction of the spring, and feel the spring force as the spring is expanded or contracted. 
The clipped spring allows the user to feel the spring force as the mouse is moved, but does not 
cause the cursor to move on the screen. Clipping is described in greater detail below. In step 
216, other types of forces can also be checked which, similar to the visual spring force, cause 

25 a dichotomy between the visual and haptic experience of the user. 

If the force sensation is not a visual spring force, then m step 218 the cursor position is 
sdi equal to the ballistic position detemiined m step 206 above. This allows flie fine 
positioning and coarse movemmt of the cursor as described above. Step 222 is then 
iaq>lemented, in which the cursor position det^mined from the above stq)s is reported to the 
30 host computer 18, as detailed below. 

If the force sensation is a visual spring force in stq) 216, then the process continues to 
step 220, where the cursor position is based on a constant scale factor. Preferably, flie scale 
factor used is the ballistic scale factor Aat was last in effect when the spring force was first 
q)plied. That is, while the visual spring force is in effect, the scaling of the reference data is 
35 held constant to the scaling that was performed just before the spring force was first output. 
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FIGURES 7a-7f demonstrate the need to provide a constant scaling during the output 
of a visual spring force. These Figures illustrate the situation of a visual spring being 
determined using reference data while the cursor is displayed using ballistic data. Figure 7a 
shows display screen 20 (display fi:ame 28) of host computer 18 displaying a gr^hical object 

5 240 for simulating a spring force. Object 240 includes a fixed portion 242 and a moveable 
portion 244, where a simulated spring betweai portions 242 and 244 is shown fully 
contracted. The user has moved cursor 1 80 onto moveable portion 244 and has held down the 
button 15 on the mouse. Figure 7b shows the equivalent positions to object 240 in the local 
frame 30. Starting point 246 is at a corresponding position in the mouse workspace to the 

10 fixed portion 242 of the grsqjhical object 240 on screen 20. The position of the mouse 12 
itself is indicated by a circle or point, which can represent, for example, axis E or the guide 
pin 78 of the embodiment of Figure 2. Mouse 12 can be positioned to the right of starting 
point 246 when the spring is fiiUy contracted, as shown; alternatively, the mouse 12 can be 
positioned on the starting point 246. 

15 Figure 7c shows display screen 20 displaying object 240 after the spring has been 

stretched by the user. As indicated by arrow 248, the user has moved cursor 1 80 quickly to 
the right. Since baUistic data is used to display the cursor on the screen, the cursor has moved 
a large distance to the right due to the fast motion of the mouse. Figure 7d shows an 
equivalent position of the mouse 12 in the local frame. The user feels a spring force pulling 

20 the mouse 12 toward the starting point, rqjresented by spring 249, where the spring force 
magnitude is based on the distance moved in the local frame 30. The mouse, of course, has 
not been moved the same actual distance in the local firame 30 as the cursor has moved on the 
screen 20 since the cursor motion has been scaled higher than the mouse motion. 

Figure 7e shows display screen 20 displaying object 240 after the user has moved the 
25 mouse 12 back toward the starting position 246 in the opposite direction to the motion of 
Figures 7c and 7d. In Figure 7e, the mouse is moved slowly by the user, as shown by arrow 
250. Thus, the cursor 1 80 does not move as £ar due to fhc ballistic algorithm scaling down the 
cursor movement to allow fine positioning of the cursor. However, as shown in Figure If, the 
mouse has been moved all the wsy back to the starting point 246. Thus, no spring force is 
30 ou^ut on the mouse 12 because the di^lacement of the mouse fcom the starting point is zero. 
However, v/hm the user looks at the screen 20, the user ejects to feel a spring force due to 
the cursor 180 still bemg some distance from the fixed portion 242 of the object 240. This 
dichotomy can be very disconcerting for the user. 

The present invention solves this dichotomy, as explained above, by fixing the 
35 mapping between the mouse and the cursor during the ou^ut of a visual spring force 
sensation, i,e., using a constant scale factor while the spring is in effect. Thus, as the cursor 
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180 is moved quickly to the rigjit away fiom fixed portion 242, the scale factor that was used 
just prior to selecting the object 240 is used and the cursor moves an qipropriate distance on 
the screen. Likewise, as the cursor is moved slowly back toward the fixed portion 242, the 
same scale factor is used, resulting in die cursor moving the same distance on the screen* In 
5 effect, the ballistics data is ignored during the output of the spring sensation. The force 
sensation and the visual motion of the cursor are thus coordinated and the dichotomy is 
eliminated. 

In an alternate embodiment, the cursor can be displayed directly according to the 
reference data (or according to a standard scale factor) rather than according to the last scale 
10 factor determined fiom the ballistics algorithm before the spring sensation was initiated. 

Referring back to Figure 6, after step 220, the process continues to step 222, in which 
the cursor position determined from the above steps is sent to the host computer 18. The 
"cursor position** is the position which the local microprocessor has determined will dictate 
the position of the cursor 180 as it is displayed by the host computer on the display device 20 

IS in the display firame 28. Hie host computer receives the cursor position and controls the 
display of the cursor at the impropriate location on the display device 20. Thus, the host 
computer simply displays the cursor as if receiving input data directly fix>m a peripheral 
device, and preferably remains ignorant of any processing performed on the reference data 
and cursor position by the local processor fiom balUstic, indexing, or other processes. This 

20 greatly reduces the processing burden on the host computer, since the microprocessor 
performs the ballistics calculations and/or the modifying of the scale factor in step 220 and 
allows the host to simply display the cursor at whatever position is rq)orted to it. The process 
200 then returns to step 204 to read another mouse position. 

Although not desoibed in Figure 6, the method 200 may also include any of the 
25 indexing features described in the present invention. In &ct, such an indexing feature is 
prefeired since method 200 makes use of ballistics, which tend to cause ofiOsets in the local 
and display fi:ames as described below. Indexing is described in greater detail with reference 
to Figure 9. The indexing described there can be adapted for method 200, by for example, 
replacing stq> 206 with steps 406-416 and adding stqps like 424 and 428 to detCTmine 
30 indexing forces if applicable. The embodiment of Figure 11 can also be adapted for method 
200. 

In addition to modifying the reference data using ballistics, indexing, or the scale 
fiictor used in step 220 before sending the cursor position to the host, the local microprocessor 
can also modify the cursor position according to other force feedback features which may be 
35 implemented in the force feedback mouse system 10. For exan^iple, clipping** can be used in 
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some situadons to puiposely rqiort a cursor position tfiat does not correspond to the mouse 
position in the local frame 30. Clipping is typically used to provide an illusion to Uxo user that 
a hard surface is being encountered with the cursor. For example, when the cursor 180 is 
moved against a wall surface, an obstruction force having a large enough magnitude force to 

5 physically stop the user's motion usually cannot be output due to actuator limitations. Thus, 
to create the illusion of a hard surface, the user is allowed to move the mouse into the wall 
against the obstruction force, but the cursor remains displayed against the surface of the wall 
as if it is impenetrable. Since the user's experience depends heavily on the visual motion of 
the cursor, an illusion of an impenetrable wall is maintained. To perform this illusion, the 

10 reference data from the mouse 12 is "clipped", i.e., modified in that the cursor position 
against the wall is reported to the host by the local microprocessor rather than the actual 
position of the mouse through the wall. Clipping may also be performed for isometric forceis 
in which a user moves flie mouse to control a rate control function of a GUI and the cursor 
ranains in a constant position. Clipping can be performed in or before step 222 if appropriate 

IS to modify the cursor position reported to the host compute. Another force feedback feature 
that can be used to modify the cursor position reported to the host computer is **(Ustuibance 
filtering." Filtering allows the local microprocessor to filter oscillations and other 
disturbances out of position data before reporting it to the host computer. This reduces or 
eliminates force-feedback-induced disturbances in cursor position that occur as a result of 

20 certain force sensations, such as vibrations. Thus, the local microprocessor can modify the 
cursor position to filter out such disturbances and rq>ort the filtered cursor position to the host 
computer. In addition, many other steps may also be involved in tihe determination and output 
of forces and the rq)orting of data to the host which are not necessary to the present invention 
and are thus not detailed herein. 

25 FIGURE 8 is a flow diagram illustrating a second embodiment 300 of the present 

invention for implementing enhanced cursor control and reaUstic force feedback in mouse 
device 10. In method 300, ballistics are provided to allow fine positioning and coarse motion 
of the cursor, and reference data or balUstic data is used for force determination to provide 
more realistic force feedback. 

30 The method begins at 302. In stq> 304, the mouse position in the local firame 30 is 

read by the local microprocessor 1 30 and is the reference position. This position is preferably 
stored in a storage area such as local memory 134 and is kq)t available for retrieval by the 
microprocessor at a later time. In step 306, a ballistic position is determined firom reference 
data. This is performed similarly to stq> 206 described with reference to Figiue 6. 

35 In step 308, the local microprocessor detemiines interactions betwera the cursor and 

the GUI (or other graphical mvironment) using the ballistic position. This step is similar to 
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Step 208 desdibed above with refecence to The microprocessor also may deteimiBe 

or dieck for events tiiat cause force sensations. In next step 310, indepradrnt forces are 
calculated, which, as described above, are those forces whidi are not based on position or 
motion of the mouse or cursor and thus require neither refermce data nor ballistic data to be 
S calculated or generated. 

In step 312, mouse based forces are calculated using the reference data. "Mouse 
based forces'' are those force sensations designated to be based on reference data in method 
300 (and 400), i.e., based on position or motion of the mouse rath^ than on the cursor. In the 
preferred embodiment, such force sensations include dampmg, inertia, and fiiction (based on 

10 velocity or acceleration). As explained with reference to Figure 6, these force s^isations are 
more realistically modelled using the reference data rather than the ballistic data. In addition, 
clipped*^ spring forces are preferably mouse-based force sensations (based on mouse 
position). Clipped spring forces differ fiom visual spring forces in that clipped spring forces 
have no visual component, i.e., the cursor does not move during the output of the spring force. 

15 Thus, no possibility of a visual-haptic dichotomy exists and the spring force can be calculated 
using the reference data. If there are multiple mouse-based force sensations to be output, then 
those forces are all determined using the reference data in step 312 and are summed to get a 
final mouse-based force. 

In step 314, cursor based forces are calculated using ballistic data from step 306. 

20 Cursor based force sensations, as referenced herein, are force sensations that arc preferably 
detemimed based on the ballistic data fix>m step 306 rather than reference data firom step 304. 
Force sensations that would cause a undesirable dichotomy between what the user sees 
visually on the display device and what the user feels if reference data were used are more 
likely to be characterized as cursor-based force sensations. For example, in the preferred 

25 embodiment, visual spring forces are designated as cursor-based force sensations, since they 
cause a dichotomy between the visual and haptic experiences of the user as e?q>lained above 
with referaice to Figures 7a-7f. When a visual spring force is calculated based on ballistic 
data, then both flie cursor position and the spring force are based on the data in the same 
di^by firame 28 and no dichotomy occurs. If there are multiple cursor-based force sensations 

30 to be output, flien all such forces are determined and summed to get a final cursor-based force. 

In step 316, the ballistic position determined in step 306 is reported to the host 
computer 18 as the cursor position. As described with reference to Figure 6, the cursor 
position is the position which the local microprocessor has detennined will dictate the 
position of file cursor 180 as it is displayed on the display device 20 in tiie display firame 28. 
35 In some situations the repotted cursor position may be a ballistic position fiirttier modified by 
clipping, distmbance filtering, or other processes as explained previously. Step 316 can be 
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perfonned at lany time once the cursor position has been det^mined, or in parallel with any of 
stq)s 308-314 and 318. 

In step 318, the forces detennined in steps 310, 312, and 314 are summed to result in a 
total force, which is then output on mouse 12 by the actuators of the interface device. The 

5 total force may include other force magnitudes contributed by other sources not shown in the 
above steps. The process then returns to step 304 to read another mouse position. Thus (as in 
all the embodiments described herein)^ all three types of forces (independent, mouse-based, 
and cursor-based) can be sunmied toge&er and output on the mouse 12 simultaneously. This 
can occur based on complex interactions and evmts of cursor and graphical environment 

10 and/or complex types of force sensations. For exanq)le, a user may stretch a line in a drawing 
program using the cursor 180, The stretching function can be implemented by outputting a 
spring force based on the distance of the stretch, and simultaneously outputting a damping 
force to slow down the mouse movement and allow better control. Thus, both a mouse-based 
force (damping) and a cursor-based force (spring) would be summed and output in step 318. 

1 S Independent forces such as jolts might also be summed with the other forces and output 

The present embodiment (and other embodiments herein) thus use both the reference 
data from the local frame 30 as well as the ballistic data for the display frame 28 in tiie 
determination of particular forces to prevent the visual-feeUng dichotomy explained above. 
The local microprocessor 130 is well-suited to keep track of data from both frames and use 

20 data from the appropriate frame as needed. For example, an enclosure command is sent from 
the host to the microprocessor 130 which defines an enclosure around a window in a GUI. 
The enclosure has force walls defined aroimd the perimeter of the window that obstruct cursor 
movement out of the enclosure. When the cursor is moved into the enclosure and is moved 
against the side of the window, the microprocessor uses ballistic data (display frame) to detect 

25 when the ciursor interacts wifli the window side. When the cursor is moved along a side wall 
of the window, a friction force is output by the microprocessor based on reference data from 
the local frame, since friction is a mouse-based force. When flie user moves the cursor to the 
comer of tiie window and stretches the window to a new size, a spring force is ou^ut by the 
microprocessor based on ballistic data from die display frame, since the spring is a cursor- 

30 based force. Thus, the microprocessor's ability to select data from di£ra:mt frames allows an 
efSdent implementation of the present invention. 

The present «nbodiment differs from the method 200 of Figure 6 in that, in method 
300, the determination of forces is modified to prevent the visual-feeling dichotomy ratho: 
than modifying the cursor position reported to the host to prevrat the dichotomy as in method 
35 200. Visual spring forces are characterized as cursor-based force sensations and are 
detennined based on ballistic data rather than a fixed mapping data or reference data. Mettiod 
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300 is more efficient for fhe local processor since ballistic data is always rq)orted to the host 
conq>uter (unless modified by other post-pnx:esses). This is unlike melhod 200, where 
refexnce data or constant-mapping data (not ballistic data) is reported to the host computer 
when oiitputting a visual spring force and ballistic data is reported to the host during the 
5 ou^ut of other types of forces, which requires the local microprocessor to monitor the forces 
and output data from a particular frame depending on the force sensation being output. In 
addition, the present method of determining a visual spring force based on ballistic data does 
not distort the spring force appreciably and any reduction in force realism is geamlly not 
noticed by the user of the mouse device. 

10 In addition, many other steps may also be involved in the detmnination and output of 

forces and the reporting of data to the host which are not pertinent to the present invention and 
are not detailed herein. 

An alternative method to that of Figures 6 and 8 for solving the visual-feel dichotomy 
involves outputting only indq}endent force sensations by mouse s;^tem 10 as described with 
IS reference to steps 210 and 310. Ballistics can be used freely in such an embodiment with no 
concern over distorting tiie force feedback, since forces are determined based only on time 
and/or other data and are not based on position, velocity, and acceleration of the mouse or 
cursor. However, in general, more realistic and inunersive forces can be implemrated using 
forces based ultimately on the position data of the mouse and/or cursor. 

20 Also, in some embodiments, both methods of Figures 6 and 8 (or 9) are available in a 

single mouse interface system, and the user may select which implemratation he or she 
wishes based on which one feels better to the user. The user may also be able to adjust the 
strength of the ballistics effect by, for example, sending parameters to fhc host computer or 
inter&ce device. 

25 FIGURE 9 is a flow diagram illustrating a third, preferred embodiment 400 of the 

preset invention for in:q>lementing ^ihanced cursor control and realistic force feedback in 
mouse device 10. Method 400 is similar to method 300 and includes a preferred CTibodiment 
of the inducing feature of die preset invCTtioiL This feature allows the user to control the 
cursor despite any limits to the mouse workspace. 

30 The indexing feature of the present invention allows a uso* to move the cursor 

throughout the display frame of the displayed graphical envirormient without causing the user 

to experience disconcerting interruptions due to the mouse colliding with physical limits to 

the mouse workspace. For example, in the described embodiment of Figure 2, the mouse 12 

may be moved in a workspace defined by the walls of guide opening 76. When the guide pin 

35 78 impacts a wall of the opening 76, a limit is reached and the user feels the collision with the 
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hard stop. Iii oth^ force feedback mouse imploneatations, hard stops may also be presCTt, 
since a mechanical linkage that transmits forces must have some physical limits to its degrees 
of freedom. The stop prevents the user fiom moving the mouse further in a particular 
direction and may prevent the user from moving the cursor to a desired target in a GUI (an 
**under-reach*' situation). These hard stops can also be very disconcerting for a user since it 
interrupts the mouse motion abruptly. 

In a traditional mouse, a user may perform inducing to re-coiter the mouse in its 
workspace aiKi reach a desired target with the cursor by simply lifting up the mouse and 
placing it closer to the center of a mouse pad or otiier area, and thrai resuming mouse 
movement. When the mouse is lifted, it stops inputting position data to the host, which 
allows the of&et between mouse and cursor to be reduced. However, the force feedback 
mouse of the desoibed embodiment cannot be indexed like a traditional non-force-feedback 
mouse by lifting up and physically re-centering the mouse in the workspace, since it is 
attadied to a mechanical linkage. An indexing mode does not address the problem of 
physical impacts: during normal use flie force feedback mouse may collide with the hard stops 
frequently, which is far more disconcerting for a user than reaching a "soft" limit to mouse 
movement on a mousepad as in a traditional mouse (where no actual impact between objects 
occurs). In addition, the hard stops are even more disconcerting and unexpected for a user of 
a force feedback mouse than a traditional mouse, since the user expects to experience hig^- 
fidelity forces based on SCTeen interactions, not a collision with an invisible stop. The fidelity 
of the force environment is corrupted by the workq)ace limits. 

One way to avoid reaching physical limits to the mouse workspace is to report only 

(scaled) reference data to the host computer, thus allowing the mouse to control the cursor to 

all limits of the screen without reaching a limit of the mouse workspace. However, as 

explained herein, such a solution does not allow the use of ballistics, which provide a greater 

degree of cursor control. Unfortonately, the use of ballistics causes the mouse position in its 

local firame 30 to become ofiBset fiom the cursor position in its display firame 28 and 

eventually causes the mouse to hit the workspace lunits. This is simply caused by the 

variable scaling of cursor portion based on mouse velocity used in ballistics. For exanq>le, if 

a mouse cmteied m its workspace is moved quickly to the right by 0.5 inches fix>m the c^ter 

point, the cursor may be moved 8 inches on the screen away fiom a screra center point. The 

mouse is (bssa moved back the same 0.5 inches very slowly and is positioned back at the 

workspace center point. However, the ciirsor is moved only 1 inch back toward the screen 

center point due to the ballistics algorithm, creating an ofifeet between the mouse and cursor 

positions in their respective fi^ies. During more movement, these ofiGsets add up, and the 

mouse may reach a physical limit to its workspace before the cursor has reached a desired 

target on the scre^ An example of such an ofi&et is shown in Figure 5 as the distance 
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between the center Q of the local fiame and the center Cs of tfie sciea (display frame). In 
such an example, the mouse can hit the physical border 401 before the cursor can reach tfie 
region 403 on the soreen. OfEsets in the local and display frames mdy also occur even what 
not usmg ballistics; for example, an spplication program or operating system may move the 
ciirsor independently of the mouse, creating an offset and requiring indexing to reduce or 
eliminate the offset. 

Thus, other methods must be used to provide a force feedback mouse that allows the 
user greater cursor control using ballistics (and to correct frame offsets that occur for other 
reasons) while also allowing an indexing-like feature to reduce offsets and prevmting the us^ 
from experiencing collisions with hard physical limits to the mouse woikspace. Since 
balUstics are implemented in the prefored onbodiments, the offsets between local and display 
frames can become large when controlling a cursor with the mouse device over time, and a 
solution is needed. Method 400 of the present invention implements such a solution by 
providing isometric limits to the mouse woikspace. 

Method 400 begins at 402. In step 404, the mouse position in the local frame 30 is 
read by the local microprocessor 130 and is the reference position. In step 406, the 
mioDprocessor checks whether the mouse is within a predetmnined distance of a physical 
limit to the mouse woiicspace. In the described embodiment, this predetermined distance is 
designated an "isometric limit" to the mouse workspace. An isometric limit of the present 
invention borders an "isometric region*' in the mouse workspace which allows a user to 
control the cursor through isometric control rather than isotonic control. As explained with 
reference to Figure 1, isometric control allows the user to control an object based on a rate 
control paradigm, where an amount of the user's pressure in a direction dictates the speed of a 
controlled graphical object in that direction. Isotonic control is the normal position control 
paradigm for mouse-cursor mapping, where the position of the mouse in its workspace 
correlates to the position of the controlled object in its woikspace. 

Isometric limits 440 and isometric regions 442 of the present invention are illustrated 
in Figure 5. Local fi-ame 30 includes a physical limit or border 444 which rq)resents the 
physical limits to movement of the mouse 12 in its workspace. For example, in the 
embodiment of Figure 2, border 444 can be the physical walls to guide opoiing 76. Isometric 
limits 440 are designated according to software (or the equivalrat) by the local 
microprocessor 130 to be at some distance d from the border 444; d can be constant around 
the bordCT 444, or d can vary at different sides or portions around the woik^ace. Isometric 
limits 440 define an isometric region 442 which causes an isometric force to be ou^ut on the 
mouse 12, as described below. The isometric region thus borders an isotonic r^on 443 
which allows normal isotonic mouse positioning. Preferably, the isom^c region 442 is an 
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edge region tibat is fairly small compared to the size of the scre»; for example, widtfi w of the 
isomdric region 442 can be 5% of screen length or width or a similar dimension. 

Referring back to Figure 9, if the mouse is not past an isometric limit 440 (i.e., not 
within the predetermined distance of isometric region 442), then the normal ballistic position 

5 of the cursor is determined. This includes step 408, in which the ballistic screen factor (BSF) 
is calculated based on the mouse velocity, and step 410, in which the change in cursor 
position is equal to ttie BSF times the change in mouse position. Then, in stop 412, the cursor 
position (which in this case is the ballistic position) is set equal to the old cursor position plus 
the change in cursor position determined in step 410. These steps are described with 

10 reference to step 206 of Figure 6. Step 418 is then initiated, as described below. 

If the mouse is past an isometric limit 440 in step 414, then the indexing feature of the 
present invention is performed. In step 414, an isometric rate is calculated based on flie 
penetration of the mouse into the isometric region. The isometric rate detemiines how fast the 
cursor is moved isometrically based on the amount of compression into the virtual spring 

15 force (explained below). The greater the distance of penetration, the taster the cursor moves. 
Thus, the isometric rate is proportional to the distance of penetration. In step 416, ttie change 
in cursor position is set equal to the isometric rate, and in step 412, the cursor position is set 
equal to the old cursor position plus the change in cursor position determined in step 416. 
Thus, the next position of the cursor is determined based on the previous position of the 

20 cursor and the position of the mouse in the isometric region 442. This is the rate control 
aspect of the isometric limits. Since the mouse has moved close to the physical border 444 of 
the woricspace, isotonic control of the cursor is no longer practical. Instead, isometric (rate) 
control is implemented based on previous cursor positions and the direction of mouse motion 
and the amount of penetration of the mouse into region 442. The movement of the cursor 

25 according to the indexing rate is described in greater detail with respect to Figure 10b. 

In step 418, the microprocessor determines interactions between the cursor and the 
graphical environment (such as a GUI) using the determined cursor position. In step 420, the 
microprocessor calculates independent forces, and in step 422, the microprocessor calculates 
cursor-based forces based on ballistic data. These steps are substantially similar to steps 308, 
30 310, and 314, respectively, of mrthod 300. 

In step 424, the microprocessor chedcs if the mouse is cuirently in indexing mode, i.e., 
whether the mouse is in the isometric region 442. If not, then the mouse is in the isotonic 
region 443 and mouse-based forces are calculated using reference data. This step is 
substantially similar to step 312 of method 300, described above. The process then continues 
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to stq) 432, described below. If the mouse is in indexing mode, then in step 428, the 
microprocessor calculates indexing forces. 

In the described embodiment, the indexing force is a resistive spring or restoring force 
that opposes the mouse's motion from the isotonic region 443 to the isometric region 442. 
The magnitude and direction of the isometric spring force is determined based on the mouse 
position within the limit region 442. For example, the spring force may have a magnitude 
based on the equation F - kx, where k is a spring constant and x is the displacement from an 
origin or starting position of the spring. In this impl^entation, k is a predetermined value 
and X is the displacement of the mouse from the limit 440 into the isometric region 442. The 
direction of the spring force is the direction opposing the motion of the mouse toward the 
physical border 444 of the mouse workspace. The isometric spring force simulates a hard 
surface against which the user may ex^ pressure to control the cursor isometrically. 

For example, FIGURE 10a shows a representation of local frame 30 in which the 
position of the mouse in the local frame is represented by circle 446. The mouse 446 is 
moved past the limit 450 into the region 442 to a position corresponding to the circle 448. 
The distance dl penetrated into the region 442 is used in the spring force equation as x to 
determine the magnitude of the opposing spring force. Thus, the further the mouse is moved 
toward the border 444, the greater the spring force opposmg motion in tiiat direction. If the 
mouse is moved diagonally toward border 444, a diagonal opposing spring force is ou^ut 

FIGURE 10b illustrates the screen 20 of display frame 28 in which cursor 180 is 
displayed as controlled by the mouse positions of Figure 14a. The dashed cursor 452 is the 
position of the cursor when the mouse 12 is at position 446. Cursor 452 is displayed far from 
the left border 454 of the screen due to offsets that have added up between mouse and cursor 
position through the use of ballistics. The diange in cursor position of cursor 452 (as in step 
416) is shown as cursor 456, which is in the same direction as the motion of mouse 12 in 
region 442 and is a distance d2 from the previous position of the cursor. The distance d2 
between the current and previous positions of the cursor is detennined by the isometric rate of 
step 414, determined by distance dl pmetrated mto the region 442 by the mouse; the greater 
the distance dl, the greater tire distance d2. Thus, the user can control the speed of tfie 
movement of cursor 452 toward border 454 by pushing agamst the spring force of region 442 
by a desired pressure magnitude that causes the mouse to penetrate flie region 442 by a 
desired distance (the distance dl is preferably used as an indication of the magnitude of input 
force from the user- the greater the displacement dl, the greater the force or pressure that tiie 
user is applying to combat the output force). As long as the us^ pushes the mouse 12 against 
the spring force, the cursor 180 will contmue to be moved m tiie appropriate direction at a 
speed controlled by the penetration distance. In this way, flie user uses isometric control to 

40 



WO99A0872 



PCT/US98/17445 



move (he cwsor on screen 20 once the physical borders 444 are reached by the mouse 12. 
Once the cursor reaches the edge of (he di^lay firame 28 in that direction, the cursor no longer 
is moved, even if the mouse is still mthin flxe isometric region 442. 

In effect, the microprocessor is changing or controlling the offset between display 
frame and local frame when determining cursor position in isometric mode. This is because 
the mouse may be fixed in position (or only being moved slightly by compressing the spring) 
in the local frame 30 while the cursor is moving in the display firame 28. As the cursor 
moves, the ofi&et between frames is reduced. Preferably, the local microprocessor stores an 
''index value** in local memory 134 as the positional ofi&et between local and display frames 
and thus may track the ofi&et between frames and may performing cursor positioning in an 
indexing situation by changing the index value along a direction corresponding to the 
direction of the penetration of the mouse into the isometric region and at a rate of change 
dq>endent on the depth of penetration into the isometric region. This is equivaloit to the usar 
in a traditional mouse system lifting up the mouse to manually perform indexing while tfie 
cursor remains fixed on the screen. It should be noted that the isometric form of control can 
be viewed as moving the entire local fi:ame 30 while the cursor stays still with respect to the 
local fi:ame. 

If the user overshoots a desired target using the isometric control, thai the user can 
immediately move the cursor in the opposite toward the target using isotonic position control, 
smce the mouse 12 will have plenty of workspace in that direction. For exan^)le, if cursor 
456 is accidentally moved past a target icon 458 so that the cursor moves to the left of the 
icon 458, then the user can move mouse 12 to the right to get the cursor back onto the icon. 
Since movement of mouse 12 to the right takes the mouse out of region 442, normal isotonic 
control of the cursor is resumed. 

In addition, y/hea using isometric regions 442, the user does not encounter a hard 
physical impact of the mouse 446 colliding with the physical limit 444. The opposing spring 
force generated in connection with r^on 442 effectively softens any movCTient toward limit 
444; and, by the time the mouse 446 gets very close to limit 444 the ^ring force is usually of 
high enough magnitude (i.e., dl is large) to repel the mouse 446 away fix>m tibie limit 444 so 
that an impact never occurs or is dramatically softened. This removes any disconc^ting hard 
coUisions when reaching limits to the mouse workspace. 

Referring to Figure 9, in stq> 430, the mioioprocessor calculates equivalent forces to 
substitute for the mouse-based forces tiiat would have been output in stq> 426 but for the 
indexing mode. This step allows forces resulting from interactions of the cursor with the 
graphical environment to be felt by the user in indexing mode. Thus, as fhe user feels the 
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spring force from the isomeric limit, the user can also feel forces ov^laid on the spring fiHce 
caused by interactions of the cursor wifli graphical objects, such as a texture force when the 
cursor moves over a **bump/' region in the GUI, or an obstruction force when the cursor 
impacts a wall. However, when the cursor is in rate (isometric) control or indexing mode, the 

5 mouse may not be moving within its workspace; the cursor moves at a rate based on the 
mouse's penetration into the isometric region, not based on the mouse's movement. Thus, 
mouse-based forces such as damping, inertia, and friction no longer make sense, since the 
mouse is not moving in a position control paradigm. Thus, an "equivalent" force to the 
desired mouse-based force is calculated in step 430 to be substituted for the mouse-based 

10 force. This equivalent force may be based on timing data or, alternatively, on cursor positions 
such as ballistic data (or other scaled data). For example, timing data may be used to 
calculate an equivalent texture force based on a frequency or duration instead of being based 
on the position of the mouse with respect to a bump or divot in the GUI. The miax)processor 
can be sent timing parameters from the host tfiat configures the time-based force sensations 

IS (the duration of a jolt force, the frequ^cy of a vibration force^ the time to start outputting the 
force, etc.) Alternatively, forces equivalent to mouse-based forces can be based on cursor 
position. A simulated divot or detent force when the cursor moves over a border of a window 
can be calculated based on ballistic (cursor) data instead of the usual mouse-based reference 
data when the mouse is in indexing mode. 

20 In next step 432, the local microprccessor reports the cursor position determined in 

step 412 to ttie host computer, and is sunilar to step 222 of Figure 6. In 5tq> 434, tiie forces 
determined in steps 420, 422, 426, 428 and 430, if any, are all sumned together and output by 
the actuators 64 on the mouse 12. Thus, flie resistive spring force is output if the mouse is in 
indexing mode as well as any forces resulting from interactions in the graphical enviromnent. 

25 The precess then returns to stq) 404 to read another mouse position. 

In an alt^nate embodiment, a hysteresis zone" may be provided between the 
isometric region 442 and the isotonic region 443 of tiie local frame. The hysteresis zone 
creates one location of the limit for ent^g re^on 442 and a difiiarent location of the limit for 
exiting region 442. For example^ in Figure 10a, when ratering region 442 with tiie mouse 12, 

30 enter limit 462 can be positioned as limit 440; and when the mouse exits region 442, exit limit 
464 can be positioned ftirther away from physical border 444. This causes the opposing 
spring force to remain active a further distance from the border 444 when exiting and 
pushes'* the mouse and cursor further from the border. This prevents the mouse 12 from 
being positioned too close to the border and allows some isotonic control of the cursor in all 

35 directions for maximum control when exiting isometric mode, i.e., the ciusor can be 

controlled in isotonic mode in a direction toward bord^ 444 for the distance d3 upon exiting 

region 442. The hysteresis zone also increases stability of the control paradigm, and can 
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enhance ttie ttansition fiom isometric control mode to isotonic control mode since the ^ring 
force will have a largo: space^range to gradually deoease in magnitude v/bea the mouse is 
exiting region 442. 

FIGURE 10c illustrates an alternate embodiment of the method 400 of Figure 9 in 
which the isometric limits as described above are intended for use as a general positioning 
mechanism for the cursor rather than as an indexing feature only for use near the edge of the 
mouse workspace. This allows the offsets between frames to be reduced/corrected and also 
prevents the mouse fix>m hitting physical limits to its woilcspace due to rate control of the 
cursor and the resistive spring force. In addition, this method would not use ballistics or otfa^ 
variable scaling of the cxusor position and thus allows realistic forces to be ou^ut. 

Reforing to FIGURE 10c, the local frame 30 includes physical workspace limits 444 
which the mouse cannot move beyond, as explained above. The central isotonic region 443 is 
defined by dashed line 445, and the surrounding isometric region 442 is defined between the 
isotonic region 443 and the limit 444. This is similar to the isometric limits of method 400 
and Figures S and 10a, except that the isotonic region 443 of Figure 10c is much smaller than 
the equivalent region of Figures S and 10a. For example, isotonic region 443 can have an area 
that is 1/4 of the workspace area. 

If the mouse is positioned in the isotonic region 443, then the cursor position is set 
equal to (or proportional to) the reference position (the cursor position may be scaled 
according to a constant scaling factor). Thus, when the mouse is in the isotonic region, cursor 
positions are directly correlated to the mouse position; no baUistic or otho- variable scaling 
processes modify the position of the cursor. If the mouse is positioned in the isometric 
region, then isometric mode is entered, which is similar to indexing mode described above. 
The cursor is positioned/moved according to an isometric rate, and an isometric spring force 
opposes the mouse's motion into the isometric region. The mouse portion preferably 
determines the speed of the cursor as it moves in the isometric region. The miooprocessor 
also overlscys any indq)endait and cursor-based forces with the indexing force as in method 
400. Thus, when the mouse is moved into the isometric region against tfie spring force, the 
cursor is moved according to a rate control paradigm in a direction corresponding to the 
direction of the mouse, where the amoimt of compression of the spring determines die speed 
of the cursor. When the user moves the mouse in the opposite direction back into the isotonic 
region, isotonic control is immediately restored and the cursor position once more 
corresponds directly to mouse position. In alternate embodimmts, a hysteresis efifect can be 
inq>lemented to provide entry and exit borders to the isometric region at different distances 
fipom the physical limit, similar to this effect described above. 
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It should be noted that, in alternate embodnnents. the isometric rate control mode 
described ^ve can be activated in other ways for indexing purposes, i.e. to position flie 
cursorafter a physical limit has been readied by the mouse. For example, instead of altering 
isometric mode when the cursor gets close to a limit, the user can sinq>ly activate isometric 

5 mode at any time by pressing and holding down a button on the mouse. Such an embodimoit 
would also preferably include spring forces on the physical limits 444 of the mouse 
workspace to soften any hard collisions between the mouse and the limits. As explained 
above, in such an isometric mode, only non-mouse-based force feedback sensations (e.g. 
based on time or cursor position) can be output, since mouse-based forces make no sense in a 

1 0 rate control paradigm. 

FIGURE 11 is a flow diagram illustrating another embodiment SOO of the present 
invention for implemoiting enhanced cursor control and realistic force feedback in mouse 
device 10. Method SOO includes an alternative embodimmt of the indexmg feature of tiie 
present invention using an edge scaling feature of the present invention to assure that an 
15 "und«*-reach** situation, where the mouse hits a physical limit in a direction while the cursor 
still needs to be moved in the corresponding direction on the screen, never occurs. 

Method SOO begins at S02. In step S04, the mouse position in the local fiame 30 is 
read by the local microprocessor 130 and is the refoence position, hi step S06, flie 
microprocessor checks wheth^ the mouse is within a predetermined distance of the physical 

20 workspace limit 444 (thus defining a predetermmcd region next to the physical limit) and 
whether the mouse is moving toward the physical limit 444. The predetermined distance can 
be a small distance or region such as 5% of the total screen dimension in that direction. If the 
mouse is not within this distance or is not moving toward limit 444, then indexing is not 
necessary and the process continues to steps 508 and 510, where the normal ballistic position 

25 of the cursor is determined by calculating a ballistic screen factor (BSF) based on the mouse 
velocity and whoe the change in cursor position is equal to the BSF times the change in 
mouse position, as in Figure 9. In step 512, the cursor position (which in fliis case is die 
ballistic position) is set equal to the old cursor position plus the change in cursor position 
determined in step 510, and st^ 522 is then initiated, as described below. 

30 If the conditions of step 506 are met, the process continues to step 5 14, where the local 

microprocessor det^mines the distance between the current position of the mouse and the 
physical work^ace limit 444 that is closest to the current mouse position, i.e., the location of 
the mouse in the predetermined re^on. In step 516, the local miopprocessor detcraiines the 
distance between the current position of the cursor and the screen limit in the display frame 28 

35 that corresponds to ihe closest physical limit to the mouse. For example, if the mouse is 
closest to the right workspace limit, the distance between the mouse and the right limit is 
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detennined in step 514 and flie distance between the cursor and tiie right screw edge is 
deteimined in step 516. If the cursor and the mouse have become ofEsdt, the distances 
resulting 6om steps 514 and 516 can be quite differmt. In stq> 518, an qipropriate scaling 
^tor is detemuned for the cursor position utilizing the distances obtained in steps 514 and 
5 516. This scaling factor is chosen to allow movement in the remaining distance in the mouse 
workspace to control movement of the cursor to the graphical Bmit of the screen 20. In other 
words, the local and display frames are recalibrated with respect to each other so that the 
mouse physical limit is not reached before the cursor reaches the screen edge, i.e., die 
microprocessor uses the offset between frames to determine a compensating scaling factor. 

10 For example, by moving the mouse to the right the user will encounter the physical 

limit 444 at a point where the cursor still has 3 inches of so-een space to trav^e in that 
direction. The local microprocessor thus will determine a new scaling factor for use in the 
predeiemiined re^on of Step 506 that will cause the cursor to reach ttie end of the screen as 
the mouse is moved through the region. This is accomplished, for example, by jGnding flie 

IS ratio between the distance found in step 516 and the distance found in step 514 to be the 
scaling factor that is multiplied by the mouse position to map the remaining mouse workspace 
to the remaining cursor screen space. 

In step 520, the change in the cursor position is detennined based on the new scaling 
factor ("edge scaling factor^'), and the process then continues to step 512 where the cursor 
20 position is detennined as the old cursor position plus the change in cursor position from step 
520. The edge scaled cursor position thus is set as the cursor position. 

In step 522, interactions between cursor and GUI are detennined; in steps 524, 526, 
and 528 independent, mouse based, and cursor based forces are detennined; in stq> 530 the 
cursor position detennined in step 512 is rq)orted to the host computer; and in stq) 532 the 

25 detennined forces are summed and the total force is output on the mouse 12. Thus, steps 522- 
532 axe substantially similar to stq)S 308-318 of Figure 8, described above. It should be noted 
^t, for cursor based forces in step 528 vAere the cursor is positioned in the edge-scaled 
region, those forces are calculated based on the edge-scaled cursor position, not ballistic 
cursor positions. For exanq>le, a visual spring force positioned in the edge scaled region 

30 would be calculated based on edge-scaled data, not ballistic data. Altematively, the method 
200 of Figure 6 can altematively be used with the indexing steps 506 and 514-520. 

Method 500 may cause some problems for the user with fine positioning of the cursor 
within the edge-scaled r^on of the screen, since the cursor motion is scaled higher in OAs 
region. However, the edge scaling is used only in the direction towards the edge of the 
35 screen. Thus, if the user ov^hoots a target during the edge scaling, the user may move the 
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mouse in fht opposite direction to acquire the target, at which point nonnal or ballistic scaling 
is used whidi typically allows easier fine positioning. 

Another advantage to the edge scaling process is that, since the cursor accelerates 
when the mouse is positioned near the limit of the mouse workspace, the user slows down 
5 motion of the mouse near the edge to compensate. This tends to duninish the likelihood that 
the mouse will hit a physical stop with high speed, and thus less of a collision force with the 
physical stops is felt. In other embodiments, the user may wish to feel the hard stops as an 
indication of the cursor hitting the edge of the screen. 

In an altemate embodiment of the method 500 of Figure 11, steps 506, 508 and 510 
10 are omitted; the distance between the current mouse position and the workspace limit in the 
direction of the mouse's movement is determined in step 514, and the distance between the 
cursor position and the screen limit corresponding to that physical Umit is determined in step 
516. This allows the local microprocessor to calculate a new scalmg factor in real time for all 
positions of the mouse in its workspace, not just for regions close to the edge of the 
15 workspace. For example, the microprocessor would always be examining the distance 
between the current mouse position and the workspace limit in step 514 and the distance 
between the cursor and the screen limits in step 516 and scaling the cursor position 
accordingly. In one example, three cursor speeds'* (i.e., cursor scalings) can be provided: 
coarse, fine, and intermediate. Coarse and fine speeds are constant mappings of cursor to 
20 mouse position allowing different degrees of control. However, the intermediate speed can 
use this altanative to method 550 to vary the scaling factor according to the offs^ between 
local and display fi'ames. In an alternative embodiment, the microprocessor can determine the 
distance of the mouse and cursor to limits on all sides, such that four different scaling factors 
can be stored and the one that corresponds to the cursor's direction is used in step 520. 

25 A differmt embodiment of an indexing feature of the present invention for avoiding 

the cumulative ofiGset between the local fi:ame and the display fi:ame is *'auto centering.** This 
method uses the actuators 64 of the force feedback mouse to automatically reduce the of&et 
between the local fi:ame and the display firames. When auto centering is to be performed, the 
local microprocessor controls tiie actuators to move the mouse to the location in the local 

30 fiame that corresponds to the cent^ of the display frame, thus eliminating tiie ofi&et between 
firames. Preferably, auto catering is pCTformed when the user is not grasping the mouse; 
otherwise, such movement would confiise the user. The auto centering can also be performed 
only when tfie offset between fimnes increases over a predetermined tiireshold. Altonatively, 
a special button, switch, or other i]:q>ut device can be provided to fiie user on mouse 12 or 

35 otfao* position which would cause the mouse to be auto centered when the input device is 
selected by the user. 
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FIGURE 12 is a flow diagram illustrating another embodiment 600 of the preset 
invCTtion for providing greater cursor control without distorting force feedback. In this 
embodiment, an alternative to traditional ballistics called adaptive resistance" is used to 
provide chanced cursor control. 

5 The method begins at 602. In step 604, the mouse position in the local frame 30 is 

read by the local microprocessor 130 and is the reference position. 

In step 606, the process examines the previous positions of the mouse to determine the 
velocity of the mouse. This can be performed similarly to the procedure that the ballistics 
steps of Figure 6 and Figure 8 to determine velocity of the mouse; for example, the velocity 
10 can be calculated or simply retrieved from a haptic accelerator or other dedicated processing 
electronics. 

In step 608, the process det^mines a resistive force magnitude that is inversely based 
on the mouse velocity. In the present embodiment, greater cursor control is provided by 
outputting forces on the mouse 12 to prevent large displacements of the mouse and thus the 

15 cursor. As in ballistics algorithms, when the user is moving the mouse slowly, it is assumed 
that the user is performing fine positioning of the cursor. Thus, for slower velocities of the 
mouse 12, a larger magnitude resistive force is output on the mouse, and for higher velocities 
of the mouse 12, a smaller magnitude resistive force is output on the mouse. In one 
embodiment, the magnitude of the resistive force can be determined using a discrete ftmction 

20 that has one or more distinct velocity thresholds. For example, if the mouse is below a 
predetermined fliresbold velocity, a first magnitude of damping resistance is selected,* and if 
the mouse is above that threshold velocity, a second, lower magnitude of damping resistance 
is selected. Alternatively, a continuous function can be referenced to provide a continuously- 
varying magnitude based on velocity of the mouse. Either a linear or non-linear function can 

25 boused 

The resistive force determined in step 608 can be any of a variety of types of resistive 
forces. For example, a damping force modelled as F = Bv, a firiction force modelled as F = f * 
(v / |vD, or a different dissipative force can be used, or a combination of two Or more forces 
can be provided The damping constant or friction coefSdent can be adjusted as described 
30 above based on the mouse velocity. Such dissipative forces slow down mid resist quick 
movement of tiie mouse so that die user will be able to position the cursor more slowly and 
accurately within the graphical environment without undesired jitters or overshooting desired, 
targets with the cursor. ConvCTsely, when the user moves the mouse rapidly, small or zero 
resistive forces are ou^ut to allow the user to perform coarse positioning of the cursor. 
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In step 610, the local micropiocessor outputs the resistive force on die mouse 12 using 
the actuators of the mouse device 10. Prefmbly, the resistive force that allows fine 
positioning of the cursor is output in all degrees of freedom of the mouse, i.e., resistance is 
felt by the user regardless of the direction of the mouse's movement. 

5 In step 612, the local microprocessor sends the reference position obtained in step 604 

to the host computer. Step 612 may be perfomied at any point in the method 600 or 
simultaneously with the other steps 606-610. Preferably, the refermce data is scaled 
appropriately according to a constant mqyping to allow mouse motions to control the cursor to 
move to all points displayed on the screen. The process then returns to step 604 to read 
10 anoth^ mouse position. 

Since enhanced cursor control is provided using forces and the mouse can control the 
cursor to all areas of the screen without hitting a physical limit, and, since reference data and 
not ballistic data is rq}orted to the host computer, the indexing features of the present 
invention are not required in the embodiment of Figure 10.. However, in some embodiments, 
15 an indexing feature may still be desked in case the local and display frames become of&et for 
some reason or to prevent the mouse from colliding hard wift a physical limit. The indexmg 
features of Figures 9 or 1 1 can be used, for example. If indexing is used, any indexing force 
may be combined with the resistive force and output in step 610. 

FIGURE 13 is a flow diagram illustrating another embodiment 620 of the present 
20 invention for providing an enhanced degree of cursor control without distorting force 
feedback. In this embodiment, force "'detents'* are used to assist the user in finely positioning 
the cursor at a desired target 

The method begins at 622. In step 624, a high level conmiand is received from the 
host computer indicating the location of detent(s) in the local frame 30 and parameters 

25 describing characteristics of the detent(s). A "detent" is a force sensation that assists a user 
in moving the cursor to a particular point or area and reduces the abiUty of the us^ to move 
the cursor away fix>m that point or area (such as a snap to'* effect). This simulates the feel of 
a physical ddten% divot, or valley in a physical surfiace. For example, when the cursor is 
moved wifliin a predetomined distance of a pomt, an attractive force can be ou^ut on the 

30 mouse 12 which he^s guide the cursor to the point Or, a spring force can be provided in a 
region surroundmg a pomt or area and be oriented in a durection toward the pomt or area, so 
fliat when flie cursor is in the surrounding region, the spring force influences the cursor 
toward the point or region. The spring force also has the effect of resisting cursor motion 
away from the point or area once the cursor has acquired the point or area. 
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Since detents assist a user in acquiring taigels, Ifaey help fhe user in fine positioning of 
the cursor. Thus a target sudi as an icon can be iniplemented as a detent with a spring force 
provided in a region surrounding the icon or a center point of the icon. It is easier for tiie user 
to acquire targets that include detents, so the need for ballistics to allow fine positioning is 
much reduced. 

In the preferred embodimrat, the host computer sends data to the local microprocessor 
indicating the locations of detmt m the graphical environment Locations of detents within 
the entire gr^hical environmmt can be sent, or just detents widun a predetennined region 
suirounding the present location of the cursor. The host also sends parameters indicating die 
magnitude of the detrat force, the shape of the detent force or shape of the region where the 
detent exists, the direction of the detent force, etc. Step 624 can be performed at any time 
during the process and can be repeated to update the local microprocessor's knowledge of the 
detents in the graphical environment. 

In step 626, the mouse position in the local firame 30 is read by the local 
microprocessor 130 and is the reference position. In step 628, the microprocessor checks 
whether the mouse is currently at a location to be affected by a force detent. The local 
microprocessor preferably checks the detent data sent to it by the host to determine if the 
mouse is positioned at the location of a detent. Thus, the microprocessor assimies that the 
cursor is positioned in the display fi:ame 28 (the screen) at a corresponding position to the 
mouse in the local fi:ame 30 according to a constant mapping. If the moiise is not ciurently at 
a detent location, the process returns to step 626 (or diecks for other interactions of the cursor 
with the GUI which may cause forces to be output). If the mouse is at the location of the 
detent, step 630 is initiated. 

In step 630, the local nucroprocessor controls the output of a detent force on the 
mouse 12 by actuators 64 according to a locally stored force model and according to any 
parametm sent by the host conq>uter. In step 632, the local microprocessors sends the 
reference position of the mouse obtained in step 624 to the host compute: as the cursor 
positioiL As described above, the xc&acmcc position may be scaled according to a constant 
(non-ballistic) mspphig. The process then returns to step 626 to read another mouse position. 

As in ttie method of Figure 12, indexing may be used in the mdhod 620 of Figure 13, 
althou^ it is not necessary since ballistic positioning is not used. Modifications for indexing 
are similar as desoibed above. 

Alternatively, the method of Figure 13 (or a similar method, such as that of Figure 14) 

can be iirq>lemented using force ^sur&ces" instead of detents to assist in fine positioning of 

die cursor. An obstruction force simulating the feel of encountering a hard surface such as a 
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wall can be provided on appropriate graphical objects or in a region around flie cursor to help 
guide tiie cursor in desired direction and/or toward a desired target For exanq>le, a force 
enclosure** can be provided around a region once the cursor is positioned within the regioiu 
An enclosure is a box-like object having sides characterized by wall and/or texture forces. 
5 For exanqjle, size, location, wall stifi&iess and width, surface texture and fiiction of the wall, 
clipping, force characteristics of the interior region of the enclosure, scroll surfaces, and the 
speed of the user object necessary to engage the forces of the enclosure can all be varied. 

Each side of the enclosure would thus resist movemrat of the cursor out of the 
enclosure and would allow the user to more easily , acquire a target inside the enclosure. In 
10 other situations, a hard surface on an object can guide the cursor along an edge toward a target 
at the end of the edge. Similarly, two surfaces forming a channel can help maintain the cursor 
on a slider bar or other linear region or object In other embodiments, a field of surfaces or an 
enclosure can be provided around the cursor, whatever its location, when the mouse moves ia 
a way to indicate fine positioning is desired, as described below with reference to Figure 14. 

15 FIGURE 14 is a flow diagram illustrating another embodiment 650 for providing an 

«hanced degree of cursor control without distorting force feedback. In this embodiment, 
force detents are provided in a region around the cursor when the user is believed to need to 
finely position the cursor 

The process begins at 652. In step 654, the mouse position in the local frame 30 is 
20 read by the local microprocessor 130 and is the reference position. In step 656, the process 
examines the previous positions of the mouse to determine the velocity of the mouse. This is 
similar to the procedure that flie ballistics steps in Figures 6 and 8 and step 606 of Figure 12 
poform to determine velocity. 

In step 658, the process determines whether the mouse velocity is less than a threshold 
25 velocity, and whether the mouse has been mder the threshold velocity for greater than a 
predetermined time period. The threshold velocity is preferably some small velocity below 
which the user typically desires to finely position the ciusor in the gr^hical envirorunent. 
The predetermined time period is prefa:ably a time p^od found to typically pass when ibc 
user is having trouble acquiring a target or performing some other fine positioiung task (and 
30 which can depend on the task). For exanq>le, a time pCTod of 3 seconds for a particular task 
might be used. In an altanative CTibodiment, oidy the velocity of the mouse is checked in 
step 658 and the time period is ignored. 

If the mouse velocity is above the threshold velocity or is not under the threshold 
velocity for the minimum time, the process returns to step 654 (of course, forces caused by 
35 other interactions of the cursor in the GUI or other events can be output as described above). 
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If the mouse velocity is less than the threshold velocity for ttie minimum time, tiien the 
process continues to step 660, where a field of multiple force detents are provided in a 
determined spacing over a determined area or re^on. Thus, the detents are not provided if the 
mouse is moving over the threshold velocity, since they would only encumb^ fast, coarse 
motion of the mouse and cursor. However, if the user is moving the mouse slowly for the 
predetermined time period, the local processor assumes that the user needs assistance in fine 
positioning, and provided the field of force detents. The detents are preferably similar to the 
detents described with reference to Figure 13, and output forces to slow quick motion of the 
mouse and cursor. The force detents can be provided in a rectangular grid, a series of circular 
radii, or in other configurations. These configurations can be predeteimined, selected by the 
user, or may vary depending on the nearest region or object in the GUI. The field of detents 
can cover the entire screen or display fi:ame, or may be provided only in a predefined smaller 
region surrounding the ciursor in a predetermined shape or a shape that differs according to the 
region or nearest object of the GUI. In addition, large detents or small detents can be 
provided, and the spacing of the detents from each other can be varied as desired. For 
example, a grid of detents can be prodded that corresponds to a grid of snap points displayed 
on the screen by a drawing program. In a word processor, the detents can correspond to letter 
spacing and line spacing of the current document. Each detent can also correspond to each 
pixel displayed on the screen. Ideally, the detents are spaced at the minimum resolution 
required for a give positioning task. For example, sensors 62 on the mouse 12 can track 1000 
points Tpcc square inch. This high resolution is not required for the host computer, since, for 
example, 300 pixels are displayed per square inch (300 dpi). Thus, detents need only be 
provided at the 300 per square inch resolution. For some tasks, detent spacing greater than 
the pixel spacing can be provided. 

The local mia-oprocessor can provide the detent field entirely independently firom the 
host computer. Alternatively, the host computer can send high level commands to enable the 
force detent feature and to characterize the detent spacing, force intensity, and other 
parameters of the detents (thus allowing the user to enable and/or characterize detents if 
desired). 

In next stq) 662, the local microprocessor checks whether the mouse is at a detmt 
location. If not, the process returns to step 654. If so, the local microprocessor controls the 
actuators 64 to output a detent force in step 664, which is similar to outputting forces in the 
above embodiments. In next step 666, the local microprocessor sends the reference position 
(or a position scaled according to a constant mapping) to the host computer as the cursor 
position (step 666 can be performed at any time after stqp 654 or in parallel with the other 
steps). 
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Method 650 is easier to implement tfian the process 620 of Figure 13, since the local 
microprocessor does not need to continuaUy be iqxlated wi& detent locations in the graphical 
environm^t fiom die host computer and does not need to check cursor position and the 
detents in the gr2q)hical environmOTt to determine if a detent is encount^^. Instead, the 
5 microprocessor need only look at mouse location in the local frame and may implement the 
detents independently of the host As with Figures 12 and 13, indexing is optional in this 
embodiment and may be provided as described above. 

While this invoition has been described in terms of several preferred embodiments, it 
is contemplated that alterations, permutations and equivalents thereof will become spparent to 

10 those skilled in the art iq^on a reading of the specification and study of the drawings. For 
example, although examples in a GUI are described, the embodiments h^in are also very 
well stiited for other two-dimensional graphical environments and especially three- 
dimensional graphical environments, where a user would like fine positioning in manipulating 
3-D objects and moving in a 3-D space. For example, the isometric limits are quite helpful to 

15 move a cursor or controlled object in a 3-D enviroiunent further than physical limits of the 
interface device allow. 

In addition, many different types of forces can be 2q)plied to the user object 12 in 
accordance with different graphical objects or regions ^ppca^ng on die computer's display 
screen and which may be mouse-based force sensations or cursor-based force sensations. 

20 Also, die various features of the embodiments herein can be combined in various ways to 
provide additional embodiments of die present invmtion. In addition, many types of user 
objects and mechanisms can be provided to transmit the forces to the user, such as a mouse, 
trackball, joystick, stylus, or other objects. Furthermore, certain terminology has been used 
for the purposes of descriptive clarity, and not to lindt the present invention* It is therefore 

25 intmded that the following appended claims include all such alterations, permutations, and 
equivalents as £dl withm the trae spirit and scope of die present mventiorL 

What is claimed is: 



52 



wo 99/10872 



PCT/US98A7445 



CLAIMS 

1. A method for providing enhanced cursor control using a force feedback interface 
device coupled to host con4>uter displaying a graphical environment and a cursor within said 
graphical environment on a display device, the method comprising: 

(a) reading a position of a manipulandum in a device workspace as a reference 
position, said manipulandum being grasped by a usct; 

(b) reporting a cursor position to said host computer derived from said reference 
position, wherein said host computer displays said cursor within said grq^hical oivironment 
on said display device at a position corresponding to said cursor position; 

(c) determining whether said cursor interacts with said graphical enviroimient as to 
cause a force to be output on said manipulandum; and 

(d) outputting said force on said manipulandum if so determined in step (c), 

wherein at least one of said reported cursor position and said output force allow said user of 
said force feedback interface device to finely position said cursor within said graphical 
environment and coarsely move said cursor as desired in said graphical environmwt without 
causing a distortion in (he quality of forces as e3q>ected to be cxperieaced by said us^ fix>m 
said force feedback inter&ce device. 

2. A method as recited in claim 1 wherein said cursor position reported to said host 
computer is said reference position that has been modified to allow said fine positionmg and 
said coarse movement of said cursor. 

3. A me&od as recited in claim 2 who^n said cursor position is a ballistics position, 
said ballistics position being said reference portion modified by a ballistics algorithm in 
which a position of said cursor is mapped to a position of said manipulandum based on a 
scaling derived &om a velocity of said manipulandum in said device workspace. 

4. A method as recited in claim 3 wherein said cursor position is said ref^ience 
portion modified by a constant ms^ing vrhen said force ou^ut in step (c) is a visual spring 
force. 

5. A method as recited in claim 3 wherein said force is determined based on motion or 

position of said manipulandum with respect to said workspace while said cursor position is 

based on previous cursor positions on said display screen. 
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6. A mefliod as ledted in claiin 5 wheran said force is d^ennined using said 
reference position if said force is based on motion or position of said manipulandum and is 
not a visual spring force, iising said ballistics position if said force is a visual spring force, and 
using neither reference nor ballistic position if said force is independent of said motion or 
position of said manipulandum. 

7. A method as recited in claim 1 whorein said force output on said manipulandum 
allows said fine positionmg and said coarse movement of said cursor wittiin said gr^hical 
environment 

8. A method as ledted in claim 7 wherein said force is a resistive force that resists 
slow movements of said manipulandum more than it resists fast movements of said 
manipulandum, said resistive force having a magnitude inversely based on a velocity of said 
manipulandum in said device woricspace to allow enhanced fine positioning while not 
debilitating coarse movonoit of said cursor. 

9. A me&od as recited in claim 7 wherein said force is a drtent force for guiding said 
manipulandum to a particular position and thereby guiding said cursor to a corresponding 
position in said graphical environment. 

10. A method as recited in claim 9 wherein said force is provided by at least one of a 
pluraUty of detents arranged in a predetemiined spacing over a predetermined area, said 
detoits being provided when said manipulandum is under a predetermined velocity. 

1 1 . A mefliod as recited in claun 9 wherein said force is provided by at least one of a 
plurality of obstruction forces simulating surfaces arranged to assist said user to controlUng 
said cursor in fine positioning tasks. 

12. A method as recited in ciaiml wherein said manipulandum is a mouse. 

13. A method as recited in claun 1 further comprising determining if an indexing 
feature should be performed, said indexing feature allowing said user to control an offeet 
between said position of said manipulandum m said woricspace and said position of said 
cursor on said display screen. 



14. A method for providing enhanced cursor control using a force feedback interfece 
device coupled to host computer displaying a graphical environment and a cursor within said 
gr^hical environmrat on a display device, the m^od comprisii^: 
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(a) leading a position of a manipulandum in a device workspace as a reference 
position, said manipidandum being graced by a user, 

(b) letting a cursor position to said host computer, wherein said cursor position is 
said reference position that has been modified to a position allowing fine positioning or coarse 

5 movement of said cursor to provide enhanced cursor control to said user when moving said 
manipulandum, and wherein said host computer displays said cursor witiiin said graphical 
enviroxmient on said display device at a position corresponding to said cursor position; 

(c) determining whether a force is output on said manipulandum based on an 
interaction of said cursor and said gnq)hical environment; 

10 (d) determining a force, wherein if said force is to be deiennined at least in part based 

on motion of said manipulandum, said reference position is used in said determination of said 
force; and 

(e) outputting said detommed force on said manipulandum. 

15 15. A method as recited in claim 14 wherein said cursor position is a balUstic 

position, said ballistic position being said reference position that has been modified according 
to a ballistics algorithm, said ballistics algorithm allowing fme positioning of said cursor at 
relatively slower movemrat of said manipxilandum and allowing coarse movement of said 
cursor at relatively fiister movement of said manipulandum. 

20 16. A method as recited in claim 15 whwein said cursor position is based on a 

constant mapping and not a ballistic algorithm when said force is a visual spring force, 

17. A method as recited in claim 15 wherein smd force that is determined at least in 
part based on motion of said manipulandum is designated as one of two types, said two types 
bdng a manipulandum-based force sensation that is based on at least one Tofetencc position of 

25 said manipulandum, and a cursor-based force s^isation that is based on at least one ballistic 
position of said cursor* 

18. A method as recited in claim 17 wherein a visual spring force is said cursor-based 
forxx sensation type, and wherein a damping force, ineitia force, and fiiction force are said 
manipulandum-based sensation type. 

30 19. A metiiod as recited in claim 15 wherein a local microprocessor, sqjarate fcom 

said host computer and provided in said force feedback interface device, stores both reference 
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positions and baffistic positions of said manipulandum over time for use in detennining said 
manipulandum-based force sensations and said cursor based force sensations. 

20. A mefliod as recited in claim 19 wberein said local microprocessor perforais said 
reporting and determining of forces. 

21 . A method as recited in claim 20 wherein said force that is not based on motion of 
said manipulandum includes a vibration, jolt or other foice sensation based solely on time 
parameters. 

22. A method as recited in claim 15 wherein said cursor position is further modified 
by indexing, said indexing aUowing control over an offset between said position of said 
manipulandum in said workspace and said position of said cursor on said display screen. 

23. A metiiod as recited in 14 wherein said cursor position that has been modified is a 
scaled reference position that allows fine positioning of said cursor, said scalmg of said 
reference position bemg performed only when fine positioning is determined to be necessary 
for positioning said cursor. 

24. A method as recited in claim 23 wherein fine positioning is determined to be 
necessary for positioning said cursor when said cursor moves within a region of 
predetermined size for longer than a predetermined time period. 



25. A force feedback mouse interfece device that provides enhanced cursor control 
over a cuKor displayed on a display screen of a host computer coupled to said interface 
device, the force feedback interfiice device comprismg: 

a mouse object physically contacted by a user and movable in a planar workspace in at 
two degrees of fireedom witii req>ect to a ground; 

a sensor Aat reads a position of said mouse object as a lefcteoce position; 

a pluraUty of actuators coupled to said mouse object fliat provide a force on said 
mouse object in said planar device work^ace; and 

a local microprocessor, separate fiom said host computer and coupled to said sensor 
and to said actuators, said local microprocessor storing said reference position in a local 
monory and 
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reporting a cursor position to said host conq)uter, ^vliaein said miooprocessor 
modifies said refermce position by a ballistics alg<xitiun to det^mine said cursor 
position, said cursor position providing enhanced cursor control to said user v/hea 
moving said mouse object, and wherein said host computa: displays said cursor within 
said graphical environment on said display device at a position corresponding to said 
cursor position, and 

outputting a force based on an interaction of said cursor with a graphical 
environment displayed by said host computer, said force being based at least in part on 
motion of said mouse object, whmin said reference position is used in said 
determination of said force. 



26. A force feedback mouse inter&ce device as recited in claim 25 wherein said local 
microprocessor modifies said cursor position by a constant scale ms^ping instead of a 
ballistics algorithm whoi said force is a visual spring force and rqports said modified cursor 
position to said host computer. 

27. A force feedback mouse interface device as recited in claim 25 wherein said local 
microprocessor uses said cursor position to determine said force when said force is a visual 
spring force. 

28. A force feedback mouse int^ace device as recited in claim 25 wherein said local 
microprocessor modifies said cursor position for indexing when said mouse position and said 
cursor position have become offset and when said mouse object is moved within a 
predetennined distance of a limit to said device workspace. 

29. A method for providing enhanced cursor control using a force feedback inter&ce 
device coupled to a host computer displaying a graphical wmronment and a cursor within 
said graphical envirorunent on a display device, flie method comprising: 

providing a manipulandum to be grasped and moved by a usei^ 

sensing the motion of said manipulandum with respect to a fixed local fi:ame and 
determming a reference position and reference velocity of said manipulandum within said 
local fi:ame; 
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deriving a scaled position firom said reference position, wheran a relation betwem 
said scaled position and said reference position is influenced by said refiaence velocity in 
Older to allow diffecent scalings for fine portioning and course positioning of said cursor, 
tho'eby providing enhanced cursor control to said user when moving said manipulandum; 

S using said scaled position in said display of said cursor within said graphical 

environment; 

determining whether a force is to be output on said manipulandum using said scaled 
position, at least in part, in said determination; 

computing a force magnitude of a force to be output on said manipulandum using said 
1 0 scaled position, at least in part, in said computation of said force magnitude; and 

outputting said computed forces to said user througih said manipulandum. 

30. A method as recited in claim 29 wherein said computing said force magnitude 
also uses, in part, said reference velocity. 

15 

31. A force feedback interface device for providing enhanced cursor control, said 
interface device being coiqjled to a host computer displaying a graphical environment and a 
cursor within said grsphical oivironment on a display device, the force feedback interface 
device comprising: 

20 a manipulandum to be grasped and moved by a user, 

a sensor that senses the motion of said manipulandum with respect to a fixed local 
fi^e and detOTnines a reference position and reference velocity of said manipulandum 
within said local firame; 

an actuator opmtive to output a force on said manipulandum; 

25 a local microprocessor coupled to said sensor and to said actuator, said local 

microprocessor 

deriving a scaled position 6om said reference position, wherein the relation 
between scaled position and reference position is influenced by said reference velocity 
in order to allow different scalings for fine motion and course motion of said cursor. 
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thereby providing enhanced cursor control to said uso- when moving said 
manipulandmn, 

reporting said scaled position to said host computer, wherein said host 
computer uses said scaled position in said display of said cursor within said graphical 
5 environment, 

determining whether a force is to be output on said manipulandum, said local 
processor using said scaled position, at least in part, in said detenxi^ 

computing a force magnitude of a force to be output on said manipulandum, 
said local microprocessor using said reference position or said referrace velocity, at 
1 0 least in part, in said computation of said force magnitude; and 

outputting said computed force to said user through said manipulandum using 
said actuator. 

32. A force feedback interface device as recited in claim 41 wherein said local 
IS microprocessor also uses said scaled position in the computation of said force magnitude. 

33. A force feedback interface device as recited in claim 39 wherein said local 
microprocessor stores an index value which is tfie of&et between the fiame of die reference 
position and the fiame of ttie scaled position. 

20 34. A method for providing an indexing feature in a forcQ feedback mouse device, 

said mouse device coupled to a host con[q>uter that displays a cursor and graphical objects in a 
graphical environment on a display screen, said mouse device including a mouse being 
moveable in a local frame, whercin said cursor is moved in a display fiame based on said 
movement of said mouse, the method comprising: 

25 detOTnining whether said mouse is within a predetermined distance to a physical limit 

of said local fiame, whmin said predetermined distance is defined by a border to a region 
positioned next to said physical limit; 

detemiining a location of said mouse in said region; 

using said location to provide control of movemoit of said cursor toward a screen 
30 limit of said display frame corresponding to said physical limit, wherein a cursor position is 
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rqrarted to said host compute that allows control of said cursor to an edge of said display 
fiame such tfiat said mouse need nev^ contact said physical limit of said W(»k8pace. 

35. A method as recited in claim 34 wherein said location of said mouse in said 
region is determined by sensing the distance of said mouse past a border of said region. 

5 36. A method as recited in claim 3S furtho: comprising ou^utting a force on said 

mouse when said mouse moves into said region, said force resisting said movem^ into said 
region. 

37. A method as recited in claim 36 wh«ein said force is a resistive spring force 
whorein a magnitude of said force is based on said distance of said mouse past said region 

10 border. 

38. A method as recited in claim 36 wherein said control of said movment of said 
cursor is provided by reporting a cursor position to said host computer, wherein said position 
of said cursor on said display screen is based on a previous position of said cursor when said 
mouse is in said region and is not based on a corresponding position of said mouse in said 

15 local fiame. 

39. A method as recited in claim 38 wherein said cursor position is also based on said 
distance of said mouse past said region border, wherein said distance detamines a speed of 
movement of said cursor. 

40. A method as recited in claim 34 wherein said step of using said location includes 
20 detomining a first distance between a current position of said mouse and said physical limit 

of said woricspace in a direction of motion of said mouse. 

41 . A me&od as recited in claim 40 further comprising deteroiining a second distance 
betwcOT a current position of said cursor and an edge to said display screen, wh^ein said first 
distance and said second distance are used in detemoining said cursor position. 

25 42. A mefliod as recited in claim 41 wherein said cursor position is based on a scaling 

of said position of said mouse, said scaling based on a ratio b^em said first and second 
distances, such that said cursor is positioned to (he edge of said screen area when or before 
said mouse reaches said physical limit to said workspace. 



43. A force feedback mouse device that provides an indexing function, said mouse 
device coiqiled to a host computer tiiat displays grq)hical objects in a gre^hical oavironment 
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on a display screen, said mouse device including a mouse being moveable in a mouse 
woikspace, wherein said cursor is moved in a soeen area based on said movonent of said 
mouse, and wherein said mouse may control motion of said cursor widiout i^aching a 
physical limit to said workspace, said mouse device comprising: 

5 a mouse object physically contacted by a user and movable in a planar workspace in at 

two degrees of freedom with respect to a ground; 

a sensor that reads a position of said mouse object; 

a plurality of actuators coupled to said mouse object that provide a force on said 
mouse object in said planar device workspace; and 

10 a local microprocessor, separate from said host computer and coupled to said sensor 

and to said actuators, said local microprocessor deteraiining whether said mouse is within a 
predetermined region adjacent to a physical hmit of said mouse workspace and detOTnining a 
location of said mouse in said region, wherein said local microprocessor uses said location to 
provide control of movement of said cursor toward a screen limit of said display screen 

IS corresponding to said physical Umit such that said mouse need never contact said physical 
limit of said workspace. 



44. A method for providing an isom^c indexing feature on a force feedback cursor 
control int^lace device, said inter&ce device coupled to a host computer that displays 
20 gr^hical objects in a grsphical environment on a display screen, said intoface device 
including a manipulandum movable in a physical workspace in order to control the position of 
a cursor displayed by said host computer wittiin said graphical environment, the method 
comprising. 

defining a border region around the outer perimeter of said physical woricspace of said 
25 inter&ce device, wherem when said manipulandum is not wittiin said border region, said 
cursor is controlled by said manipulandum throug^i a position control paradigm and when said 
manipulandum is wi&in said border region, said cursor is controlled tfarou^ a rate control 
paradigm in at least one direction of motion; 

determining whether said manipulandum is within said bordo* region and determining 
30 an amount of penetration into said bonier region; 
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ou^utting a force on said manipalandum opposing said penetration into said boid^ 
region, a magnitude of said force being based on the dq>tfa of said pen^tion into said hotdsr 
region; and 

using said penetration into said border region to control a speed of movmient of said 
5 cursor along a particular direction on said display screen. 



45. A method as recited in claim 44 fiirther comprising implraienting a hysteresis 
efifect by removing said force on said mouse whai said mouse is moved away from said 
physical limit, wherein said force is removed at a distance greata: from said physical limit 
10 than said distance defining said border to said region used for ou^utting said force. 



46. A method for providing an isometric indexing feature in a force feedback cursor 
control interface device, said interface device coupled to a host computer that displays 
gr^hical objects in a gr^hical environment on a display screen, said interface device 
15 includmg a manipulandum bdng movable in a physical workspace in order to control the 
position of a cursor displayed by said host computer within said graphical oivironment, the 
method comprising: 

defining a local frame and a display frame, wherein a location of said manipulandum 
in its workspace is referenced with respect to said local frame and wherem the location of said 
20 cursor with respect to said gr^hical environment is referenced with respect to said display 
frame; 

providing an index value that defines a positional ofi&et between said local fi^e and 
said display fi:ame; 

defining a bonier rpgion around an outer perimet^ of said pl^^cal workspace of said 
25 inter&ce device; 

determining whether said manipulandum is within said border region and detennining 
a penetration into said border region; 

ou^utting a force on said manipulandum opposing said penetration into said border 
rpgion, a magnitude of said force being based on a depth of said penetration into said border 
30 r^on; and 
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using said praetration into said border region to modify said index value along a 
direction corresponding to a direction of said penetration and at a rate of change depeadeat 
upon a depth of penetration into said border region. 

5 47. A method as recited in claim 46 wherein a rqjresentation of said local fiame» said 

display frame, and said index value is stored in memory accessible to a processor local to said 
interface device and sqiarate from said host conq)Uter. 



48. A method for providing an indexing function in a force feedback mouse device, 
10 said mouse device coupled to a host computer that displays grq>hical objects in a gn^hical 

environment on a display screen, said mouse device including a mouse moveable in a mouse 
workspace, wherein said cursor is moved on a display soreen based on said movement of said 
mouse, the method con^>rising: 

determining a mouse distance between a current position of said mouse and a physical 
1 S limit to said mouse woikspace; 

determining a cursor distance between a current position of said cursor and a displayed 
edge of said display screen; 

detennining a scale factor based on said mouse distance and said cursor distance and 
scaling a position of said mouse; 

20 scaling said position of said mouse to detemune a cursor position and reporting said 

cursor position to said host computer, said host computer displaying said cursor at a position 
on said display screen corresponding to said cursor position, said cursor position allowing 
said cursor to be positioned to an edge of said display screen when or before said mouse 
reaches said physical limit to said woikspace. 

25 

49. A method as recited in claim 48 further comprising detennining whether said 
mouse is within a predetermined region a<y acent to a physical limit of said mouse workspace, 
wherein said scaling of said mouse position is performed only when said mouse is within said 
predetomined region. 

30 
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